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I. INTRODUCTION 
Harotp A. EDGERTON AND WILLIAM J. JONES 


Learning by doing is a frequently indorsed 
but seldom practiced principle of teaching at 
the college level. While it is not uncommon 
for teachers to lecture for hours on the value 
of direct experience, few actually carry on in- 
struction in their classes in terms of real 
problems. The values which accrue from such 
direct experience cannot be evaluated solely 
in terms of the products of such experiences 
for the very act of participating in the 
Processes involved in direct experience has 


unique values which need to be considered in 
judging the worthwhileness of learning by 
doing. 

The series of articles which follow are the 
products of a class in statistics at Ohio State 
University which tackled real problems in 
terms of actual data and arrived at conclu- 
sions which were of importance to college 
officials. These articles do not constitute 
even the principal source of data for evaluat-. 
ing the instructor’s hypothesis that students 
are likely to become better statisticians if 
they work on practical problems, but they 
are offered as partial evidence of the worth 
of such an approach to the teaching of 
statistics. Other factors which need to be 
considered in appraising the value of this 
“problem method” of teaching statistics are 
the manipulative skills which were developed 
with respect to the use of the Hollerith ma- 
chine, the techniques of setting up procedures 
for collecting and organizing raw data, the 
administrative techniques of actually organ- 
izing and carrying through a study on sched- 


ule, and the ability to write an account of 
the study. 


The following articles should be of inter- 
est not only because they are the product of 
a class, but because they constitute a rather 
comprehensive study of a series of five evalu- 
ation devices which were used in conjunction 
with an orientation course for college 
freshmen. 


In 1937, the College of Education at the 
Ohio State University inaugurated an experi- 
mental freshman orientation course called 
Survey of Education 407. One of the prin- 
cipal assumptions which operated to direct 
the organization of the course was that the 
personnel work and the instructional activity 
of the College needed to be more closely in- 
tegrated. Accordingly, all of the students 
taking the course were assigned on the basis 
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of their proposed teaching field interests (if 
any) to advisory groups of about 15 to 20. 
The students met twice a week in their small 
advisory sections and twice a week in much 
larger lecture sections. 

In order to facilitate the guidance work 
which was carried on and in order to provide 
the students with more information about 
themselves with respect to a wide variety of 
aspects of personality, several testing devices 
were given to the students during the course. 
Since the objectives of the college program 
include a concern for interests, attitudes, 
habits of thinking, and several other aspects 
of personality, in addition to subject matter 
accomplishment, many evaluation instruments 
designed to measure such objectives were 
made features of the testing program of the 
course. 

A veritable wealth of information was thus 
available, and constituted the raw data for 
the studies which follow. The instructor and 
members of the class acknowledge the kind- 
ness of the office of the Junior Dean of the 
College of Education and the Evaluation Di- 
vision of the Bureau of Educational Research 
for their cooperation in making the data for 
these studies available. 

Five evaluation instruments were singled 
out for study by the members of the class 
as follows: ; 


. Contemporary Affairs Test (1938 form) 
. Interpretation of Data Test 

. Interest Questionnaire 

. Social Problems Test 

. Social Attitudes Scale 


mewn eae 


All of these testing devices, with the ex- 
ception of the Contemporary Affairs test, 
were developed by the Evaluation Staff of 
the Eight Year Study of the Progressive Edu- 
cation Association. The Contemporary Af- 
fairs test was developed by the Cooperative 
Test Service. Each of these tests is described 
in the studies which follow. 

A unique feature of the following articles 
is that they all have exactly the same in- 
dividuals included in the statistical samples 
used. The 462 cases which constitute the 
sample for one study make up the sample 
for every other study in the series. In fact, 
the 462 cases for these studies were selected 
on just such a basis of communality. In ad- 
dition to the test data mentioned above, as 
much of the following information was col- 








[Vol. 5, No. 2 


lected about each of the 524 autumn quarter 
1938 freshmen as was possible by consulting 
available university records: 


. Age at time of entrance to college 

. Size of high school graduating class 

. Rank in high school graduating class 

. Father’s education 

. Mother’s education 

. Father’s occupation 

. Best liked high school subject 

. Least liked high school subject 

. Ohio State University Psychologica) 
Examination. 

. Reading test score on Ohio State Upj. 
versity Psychological Examination 
11. English placement test score 

12. Troyer Index* 

13. Autumn quarter point-hour ratio 

14. Winter quarter point-hour ratio 

15. Cumulative point-hour ratio 

16. Grades in various courses 

“An average of the student’s reading percentil: 


gence test percentile, and an application blank ; 
based on 19 factors. 


Oo wor Ou PWN 


Oo 


When all of the above data for the entire 
group of 1938 autumn quarter freshmen were 
collected, they were examined for complete- 
ness. With respect to items 14 and 15 above 
the data were found to be more incomplete 
than they were with respect to the other factors 
listed. Although from the outset it seemed 
desirable to the class to work toward the end 
of having an interlocking set of studies based 
on identical cases, and although it would 
have been desirable to have more common 
data such as those listed above to relate to 
the test variables, yet it seemed that 4 
definite bias would have been introduced into 
the sample if all those cases were rejected 
from inclusion in the sample for which there 
were no winter quarter or cumulative point! 
hour ratios available. A definitely scholasti- 
cally inferior group of students would have 
been thus rejected, and the remaining sample 
would not have been as typical of entering 
autumn quarter freshmen. Thus, while the 
criteria for inclusion in the sample which was 
finally selected could have been chosen 1 
terms of completeness of the data with 
respect to most if not all of the sixteen lac 
tors listed above, (and it would have beet 
desirable to have had more such common 
formation to correlate with the test vat 
ables), yet in the interests of selecting 4 
typical sample, fewer criteria were applied. 
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F Only those cases have been rejected from the 
sample for which intelligence test data, 
autumn quarter point hour ratios, and scores 
n all five of the tests which were selected 
were incomplete. Four hundred sixty-two of 
the cases had complete data for these seven 
factors and thus constitute the common 
sample for all of the studies which follow. 


In order to characterize better the sample 
ysed in these studies, it may be noted that 
‘the 462 students, 305 were girls and 157 
were boys. This is a ratio of girls to boys 
of almost exactly two to one. A study of the 
racial composition of the group indicated 
that the sample contained 437 white cases 
and 25 colored cases, or 94% white and 
6% colored. 

Table I shows the distribution of ages, 
measured from the last birthday, at the time 
of entrance to college of the students com- 
posing the samples. It should be noted that 
over half of the group was 18 years old at 
the time of entrance to college. Further evi- 
dence of the age homogeneity of the group is 
the fact that 85% of the group was between 
i7 and 19 years of age at the time of entrance 
to college. 

One way to characterize the size of the 
communities from which students come is to 
use the size of the high school graduating 
lass as a rough index, for usually the larger 
graduating classes are found in the larger 
communities. Mid-year graduating classes 
and consolidated schools’ classes, however, 
are exceptions to the generalization. Table IT 
indicates the size of the graduating class of 
the students composing the sample. It will be 
noted that the group is not homogeneous in 


TABLE I 


Ace DISTRIBUTION OF 462 OHIO STATE 
UNIVERSITY EDUCATION FRESHMEN 
AT TIME OF ENTRANCE 


Autumn, 1938 


Year of Birth Number Per Cent 
24 or older 
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TABLE II 


S1zE OF HIGH SCHOOL GRADUATING CLASS OF 
AUTUMN QUARTER EDUCATION FRESHMEN 


Size of Class Number Per Cent 
631 or more __ ne 3 1 
399-630 _ _ 1M oe , 73 16 
ne 62 13 
159-251 : = 46 10 
100-158 _ : eas 48 10 
64— 99 __ ms 8 
40- 63 _ : 10 
26— 39 3 d s 
16— 25 ‘ \ 
10- 15 - ‘ 5 
No data - 11 


Totals 100 


this respect, for the distribution has high 
variability. The median size of high school 
graduating class was 100, while the step in- 
terval having the largest frequency is 399— 
630. These data seem to point to the fact 
that the students come from schools with 
large graduating classes, and probably from 
large communities more often than from 
small or rural communities. 

Over one-half of the freshmen studied 
ranked in the upper third of their high school 
graduating classes, while 33% ranked in the 
middle third, and 12% ranked in the lower 
third. 

Table III presents the extent of the fathers’ 
and mothers’ education of the students who 
were studied. Slightly more than one-half of 
both the fathers and mothers have an edu- 
cation equivalent to high school education 
or better although about three-fourths of the 
parents have not attended college. More 
fathers than mothers have been graduated 
from college, the figures being 16% and 5% 
respectively. Approximately one-fourth of the 
students’ fathers and mothers have had only 
eighth grade education or less. The two-way 
table likewise reveals that nearly half the 
fathers and mothers have had the same 
amount of education and that in cases where 
the parents have had unequal amounts of 
education, the father has had more education 
than the mother two times out of three. 
Column 1 in Table III reveals the interesting 
marital combination of a woman college 
graduate with a man who had no more than 
eighth grade education. 

The distribution of fathers’ occupations ac- 
cording to the United States Employment 
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TABLE III 


EXTENT OF FATHERS’ AND MOTHERS’ EDUCATION OF 462 OHIO STATE UNIveRsity 
COLLEGE OF EDUCATION FRESHMEN 


Autumn, 1938 


Fathers’ Education ( 
(1) Collese Graduate. —._._.......----...... 1 
(2) College, but not Graduate —- Sa colviiaide 
(3) High School Graduate -------------~--- 
(4) High School, but not Graduate ---_-~-- 


(5) Eighth Grade or Less ~--------~- 
(6) No data ~.....-.....-....----------.- 


Totals 
Per Cent —--- " - ara iestersaiaed 


Service’s classification is shown in Table IV. 
The data indicate that the students come 
largely from the professional, sales, and 
skilled craftsmen classes of workers, nearly 
two-thirds of the fathers being classified in 
these groups. 

The distribution of the intelligence test 
scores of the education students who were 
made the objects of the studies which follow 
is presented in Table V. The skewness of the 
distribution toward the upper end may be 
attributable to the fact that until recent 
years a selective admission policy was opera- 
tive in the College of Education with an in- 
telligence percentile rank of 30 being regarded 
as a critical point. While this selective ad- 
mission on a basis of intelligence is no longer 
in force, secondary school officials may yet 
be making it one of the bases for recom- 


TABLE IV 


FATHERS’ OCCUPATIONS OF 462 OHIO STATE 
UNIVERSITY COLLEGE OF EDUCATION 
FRESHMEN 


, Autumn, 1938 


(According to the United States Employment 
Service Classification) 


Occupation Number Per Cent 
Professional and kindred __ 108 23 
Sales persons ____________ 93 20 
Clerical workers __________ 35 8 
Service workers __________ 26 6 
Skilled craftsmen’ ________ 113 24 
Production workers _______ 27 6 
Physical laborers _________ 25 5 
Unassigned ______________ 9 2 
ER 26 6 

0 462 100 


*Includes farmers. 
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Mothers’ Education Totals Per Cen: 
(2) (3) (4) (5) 
26 19 5 3 5 73 16 
23 «15 5 7 0 52 11 
16 74 13 13 3 121 26 
8 10 30 19 4 73 16 
i Z2 19 +64 3 116 25 


2 3 2 3. #14 27 5 


82 1438 74 109 29 462 
18 31 16 24 6 100 
TABLE V 


DISTRIBUTION OF INTELLIGENCE TEST PERCEN- 
TILES FOR 462 Onto STATE UNIversity 
COLLEGE OF EDUCATION FRESHMEN 


Autumn, 1938 


Percentiles Number Per Cent 


oe 75 16 
_ SS eee 62 13 
nc a ii 65 14 
ON a 64 14 
ee ENS ee ae ee 45 10 
J SSE ee 46 10 
LE Betecnce sta eee eae 41 9 
ASI a 21 5 
| eee eee ee 25 5 
ene, ele ena = 18 4 
0 ee - ~@62 100 


mendation to college, and thus a once- 
removed selective admission policy might be 
in effect. It should be pointed out, too, that 
the only cases which were included in the 
sample were those for which autumn quarter 
point hour ratios were available, and that the 
application of this criterion served to reject 
from consideration many students wh 
dropped out of school before the end of the 
first quarter. Probably more students with 
low intelligence test scores than students wit) 
high intelligence test scores dropped out be- 
cause they were unable to meet scholastic 
requirements during the first quarter. This 
may be another possible explanation of the 
skewness. A third possibility is that the 
changing character of entering freshmen trom 
year to year has rendered the norms less 
stable and less comparable. It is probably 
true that the sample selected for study was 
not different from the entire autumn quarter 
education freshman group of 1938. 
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1.A STUDY OF A _ TEST FOR 
MEASURING SKILL IN THE 
INTERPRETATION OF DATA 


Wape S. AMSTUTZ 
RupERT C. KOENINGER 


The Interpretation of Data test used in the 
freshman testing program was designed to 
test the students’ ability to analyze data and 
vet meaning from them. The test consists of 
ten sets of problem data about which the stu- 
dent is asked to make interpretations. Each 
set of data has a variety of statements which 
the students are asked to interpret as true, 
probably true, insufficient data, probably 
false, and false. To give a correct answer to 
all these involves skill in recognizing state- 
ments, summarizing the data, contrasting 
portions of the data either by regrouping the 
data or by translating the relationships into 
easier terms, and recognizing high and low 
points as well as points of change. To do 
this involves an ability to recognize the 
tacts. 

The test is also designed to measure ability 
in drawing interpretations and _ projecting 
meanings. What meanings have the facts 
presented in the problems? One set of con- 
clusions are somewhat beyond the facts but 
in general are in agreement with a definitely 
established trend. Another set of inferences 
concern points not actually presented in the 
data and, at the same time, are consistent 
with an established trend. In the former we 
have “prudent extrapolation” while the latter 
involves “prudent interpolation”. 

A third characteristic is concerned with 
the ability to recognize the limitations of the 
data under consideration. There are instances 
in which there is insufficient evidence to war- 
rant a given statement. If a statement is 
made and not supported by the evidence, the 
student is going beyond the available data. 
A more detailed analysis shows five kinds 
ol statements in the test. The first type are 
those which are rather fully supported by the 
data. They may be statements which sum- 
marize the data, statements which indicate 
trends, statements which make new groupings 
of data, or translate relationships into easier 
terms or ratios, statements which point out 
significant items, or statements which indicate 
the range of the data. All of the above types 


of statements should be marked “true” by 
the students. 
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The second type of statements are those 
which may be thought of as only “probably 
true” and which are partially supported by 
the evidence. Included in these are state- 
ments which are slightly beyond the facts but 
are consistent with all that is known about 
the data, statements which involve points 
within a trend but are not established by the 
data, and those which draw conclusions about 
larger groups on the basis of adequate 
sampling. 


The next type are statements which involve 
interpretations which are considered as “un- 
certain” because of the insufficient evidence. 
Included are statements which assign cause, 
predict effects or assign implication to the 
data, attribute purpose, extrapolate the 
trends too far, attribute values to outcomes, 
contain false analogies, make unjustified gen- 
eralizations, or make use of a “per cent- 
equals-number”’ fallacy. 


In the fourth category are those statements 
which are considered “probably false”. In 
this group are found statements which go 
slightly beyond the facts and are inconsistent 
with the data, those which involve points not 
established by the data, which are incon- 
sistent with a trend, and those statements 
which draw conclusions about larger groups 
which are contradicted by a sample. 


Finally, there are those statements about 
the data which are “false”. Included in these 
statements are those which are inaccurate 
summarizations of the data, contradict a given 
trend, make false contrasts and erroneous 
computations, present inaccurate ranges and 
are contrary to the evidence, contrast one 
trend with another inaccurately, and those 
which group the data erroneously. 


Faced with statements of the kinds indi- 
cated, it was the student’s task to mark each 
of them as either true, probably true, in- 
sufficient data, probably false, or false. With 
such a set of student responses, eight differ- 
ent kinds of scores were obtained on the test. 
Hereafter they will be referred to as variables 
I to 8. 


(1) General accuracy 

(2) Accuracy with probably true and 
probably false statements 

(3) Accuracy in recognizing statements 
which have insufficient data to support 
them 
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(4) Accuracy with true and false state- 
ments 

(5) Score indicating omissions 

(6) Caution 

(7) Beyond the facts 

(8) Crude errors 


The general accuracy score (variable 1) 
indicates the students’ ability to mark all 
five types of statements correctly. This score 
is obtained by totaling the number of insuffi- 
cient evidence items marked uncertain, the 
true items marked true, the probably true 
items marked probably true, the probably 
false items marked probably false, and the 
false items marked false. (In this study all 
scores are given in per cent of possible score 
in that classification.) 

The second score indicates the general ac- 
curacy with which the student judged only 
those statements which are probably true or 
probably false. The third indicates the stu- 
dents’ ability in correctly recognizing state- 
ments which are not supported by the evi- 
dence available. The fourth is a score of the 
per cent of true and false statements which 
a student marks correctly. Provision is made 
for a fifth score indicating the number of 
omissions students made. 

The sixth is a caution score indicating to 
what extent the student had a tendency to 
judge statements with less certainty than the 
data warrant. If a student marked a true 
statement as probably true, he is scored 
“cautious”. If he marked a probably true or 
probably false statement as insufficient data, 
he was likewise scored “cautious”. 

A common type of student error is that of 
going beyond the facts as presented in the 
data. The seventh score indicates the extent 
of this kind of error. If a student marked a 
probably true or probably false item as true 
or false, he was going beyond the data. The 
same is true if he marked any of the insuffi- 
cient data statements as true, probably true, 
false, or probably false. 

The eighth score is called crude errors and 
gives an indication of the students’ tendency 
to make wide errors in judgment. For 
example, if a statement is marked false when 
the information would support it as true, or 
true when the information would indicate it 
as false, he is making a crude error. 

Table I presents a summary of the vari- 
ables in terms of their sources. For example, 
Table I shows that if a true statement were 
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TABLE I 
A SUMMARY OF THE SCORING Scuewe 


Statements which 


should have been If marked by this 


marked student 
T PT ID PF 
[ aE 6 6 8 
PT Se oe ee en OTE 7 (2) 6 » 
ID wae P| (3) 7 
> SS 8 6 (2) 
ee ee ee 8 s 6 6 


* Those variables in parenthesis 
to the general accuracy score as 
those indicated by the number. 


marked true, it would contribute to the scorn 
on variable 4, a correct response. If a try: 
statement were marked probably true \ 
sufficient data, it would contribute to th 
score on variable 6, a caution error, and 
it were marked probably false or false, i: 
would contribute to the score on variable ‘ 
a crude error. 
It is believed that true and false statems 

are primarily ones that require only 


reading habits. One can look at the data and 
see whether or not the statement is true or 


false. The probably true and probably {als 
statements present a slightly more difficul 
situation. There are two main usages of the 


“probably true” statement. One of these is 


the projection of a trend, either forward or 
backward, and the other is a statement oi 
sampling. If it is true of this particular group 
in question, then it is probably true of an- 
other or a larger group. In short, the range 
of statements that are possible is limited in 
the area of probably true and probably ‘ake 
statements. With the insufficient data state 
ments the situation is different. There ar 
many ways of going beyond the data. for 
example, one may attribute cause, propo 
value judgments, or assign implications 0 
cosmic purposes. In the trials with the test! 
was found that there were more of this |: 
ter type of statement needed in the test 9 
order to make the test more reliable. 
The reliability of the test as determined » 
giving it to 110 tenth grade students in three 
different schools is shown in Table Il 
The estimated reliability for this particus! 
sample and the number of possible items 
each variable of the test are also included 
Table II. It is to be noted that the fifth van 


*Data furnished by the Evaluation Staff f ; 
Year Study, Progressive Education Association, 1 ¥¢ 
Chicago, Chicago, Illinois. 
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TABLE II 


beTIMATED RELIABILITY COEFFICIENTS OF THE 


SeVERAL SCORES ON THE INTERPRE- 
TATION OF DATA 


Number 10th 
of Grade Estimated 


Variable Possible Coeffi- Coeffi- 
Items cient cient* 
1 General accuracy 119 91 81 
) Probably true and 
probably false. 29 51 67 
Insufficient data_ 57 91 86 
$ True—False _--. 33 85 65 
5 ()y ssions — 119 om 19 
& Caution ...._._. 62 an .68 
7, Beyond facts —-- 86 88 .85 
8 Crude errors _-. 62 87 45 
Fstimated by means of formula of Case IV 


developed in the article, Kuder, G. F., and 
Richardson, M. W., “The Theory of the Esti- 

ition of Test Reliability’, Psychometrika, 
Vol. 2, Sept. 1937, pp. 151-160, Case IV. 


ble is omitted from the tenth grade study. 
Frequently this variable is no longer used, as 
the general opinion seems to prevail that 
little if any pertinent information about the 
student is reflected by the number of his 
missions. 

The purpose of this inquiry is to study the 
Interpretation of Data test in terms of the 
intercorrelations of the eight scores made by 
the 462 College of Education freshmen. Is 
there a high or low correlation between this 
test and the scores made on the intelligence 
test? Does a significant correlation exist be- 
tween the scores made on the Interpretation 
of Data test and the grades the students made 
for the fall quarter? Is it possible that two 
parts of the test are measuring the same 
thing? Would the same purpose be served by 
an elimination of some parts of the test? In 
view of the increased time required for ad- 
ministration and scoring of this test, it would 
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be an economy of time and labor if the same 
information could be given by three or four 
variables instead of seven or eight. 

Certain statistical characteristics of each 
of the variables are shown in Table III. A 
casual examination of the scores on the parts 
of the test as shown in Table III indicates 
that some of the distributions are noticeably 
skewed. Coefficients of skewness were com- 
puted for all the test variables, using the 


_3(M — Md) 


formula sk = . All of the test 


o 
variables showed positively skewed distribu- 
tions except variables 1, 3, and 4. None of 
them, however, were what one would call 
““much skewed’’. 

The intercorrelations of the test variables 
and their correlation with four of the “com- 
mon data” variables listed in the introduction 
to this series of articles are shown in 


Table IV. 


In the correlations shown in Table IV, 
three stand much higher than the others. 
Variables 1 and 3 correlate 0.78; variables 1 
and 7 correlate — 0.77; and variables 3 and 
7 correlate — 0.88. Since variable 1 is the 
sum of variables 2, 3, and 4, a high positive 
relation is expected of that variable with its 
component parts and a negative correlation 
of perhaps some lesser magnitude with the 
other variables. This tendency should be 
present since it is desirable to have a high 
score on variables 1, 2, 3, and 4 and to have 
a low score on variables 5, 6, 7, and 8. 

It is interesting to note in Table IV that 
the correlation of variables 10, 11, and 12 
(intelligence test score, point hour ratio, and 
grade in Survey of Education) show almost 
the same correlation with each of the test 
variables. The greatest difference between 
any two r’s in a given column is 0.06. This 


TABLE III 


AVERAGES, DISPERSION, AND SKEWNESS OF VARIABLES 


N = 462 Education Freshman Students 


Test Variable 
General accuracy 
, Probably true and probably false 
- insufficient Data 
True—False 
o Omissions 
. Caution 


to, 


tec 


* 





Standard 
Mean Median Deviation Skewness 

sie 47.6 47.8 10.2 —.06 
pesaiaiveni 25.8 25.3 ph hy 11 
siete 48.6 49.3 16.8 —.13 
eins thane 59.9 61.4 14.1 —.32 

see 2.6 2.1 3.1 48 
ee ee 28.0 27.9 10.0 .03 
selapaticlos 44.9 44.4 13.4 11 
mentibeibtot 14.6 13.8 6.0 40 
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TABLE IV 


TABLE OF INTERCORRELATIONS OF THE TEST VARIABLES AND CORRELATIONS OF TEST 
VARIABLES AND CERTAIN REFERENCE VARIABLES 


254 
Variables 1 2 
1. General Accuracy 31 
2. Probably True and 
Probably False 31 
3. Insufficient Data_ _- .78 .03 
4. True—False______- . 61 Pe 
5. Omissions. _ _ - a7 .10 
6. Caution.......- 07 . 28 
7. Beyond Facts__--__- —.77 .08 
S truce........-- . 56 .15 
9. Rank in High School 
Graduating Class 21 .09 
10. Psychological Test . 48 . 26 
11. Point Hour Ratio 
(Autumn Quarter) _- . 49 . 25 
12. Survey of Education 
(Required course) . 46 . 25 


raises the question whether or not these three 
variables are measuring the same thing. Re- 
moving intelligence (variable 10) from the 
point hour ratio (variable 11) the residual 
was correlated with each of the test variables. 
These part correlations are shown in Table V. 
For the purpose of comparison the total cor- 
relations with point hour ratio are given. The 
data contained in Table V indicate that in- 
telligence and point hour ratio have a very 
great similarity, but the variables are not to 
be considered statistically equivalent. 
Another great similarity in coefficients oc- 
curs with respect to variables 3 and 7. By 
further examination of the test it is found 
that the student has only two possible an- 
swers for all “insufficient data’’ statements. 
By definition he will either answer it correctly, 
or incorrectly as “beyond the facts”. Should 
one decide to remove one of these variables 
on the assumption that they measure the 
same thing, g contradiction may exist. For 


TABLE V 


CORRELATION OF THE TEST VARIABLES WITH 
PoINT Hour RATIO (INTELLIGENCE TEST 
ScoRE REMOVED FroM Point Hour Ratio) 


Part Total 


Variable Correlation r 

1. General Accuracy ______ .255 494 
2. Probably True and Prob- 

ably False ~_________ 124 .258 
3. Insufficient Data _______ .145 .318 
4. True-False ____________ .239 443 
5. Omissions _____________ —.042 —.055 
SS —.199 
7. Beyond Facts __________ —.141 —.305 
8. Crude Errors __________ —.195 —.391 





3 4 5 6 7 

78 . 51 —.17 .07 % 

-.03 «42 -~10 —.28 O8 13 
.15 .14 45 88 4 

15 .15 55 07 5] 

.14 —.15 4 04 f 

.45 . 55 mm 49 \4 

88 -.07 —.04 ~. 49 1] 

_39 . §1 -07 —.04 37 

26 .31 -. 09 —.08 22 

.33 .41 -. 03 —.16 32 

. 82 . 44 .05 20 31 

.30 .38 07 .10 30 


example, if variable 7 (beyond the facts 
were removed it would be impossible to score 
variable 3 “insufficient data” when the stu- 
dent answers it incorrectly. Furthermore, 
since there are only two possibilities with this 
type of test item, i. e., correct or beyond the 
facts, one can represent this relation by a 
linear function and predict exactly one of the 
scores when the other is given. The reason 
the correlation between variables 3 and 7 is 
not perfect is due to the fact that there is a 
small probability for an answer beyond the 
facts in the probably true and probably false 
categories. If the probably true and proba! 
false statements were removed from the test 
the result would be a perfect correlation be- 
tween variables 3 and 7. 

Is it necessary to use all eight scores t 
have the essential information contained in 
the test as now scored? Considering the fact 
that there are several high correlations, '! 
would seem that variables 1, 3, and 7 reflect 
essentially the same information. These three 
variables also have somewhat similar correls- 
tions with other variables. 

Since no other such combinations were 4p- 
parent from casual inspection, factorial ana'y- 
sis was applied. Using Thurstone’s centroic 
method, the loadings for four factors wert 
obtained. These are shown in Table VI. 

The three variables 1, 3, and 7 stand out 
in the first factor with loadings of 0.945, 
0.869, and — 0.837 respectively. To the 
might be added variable 8 (crude errors, 
with a loading of —o.603. Thus the ts 
centroid factor is made up largely of variable 
1, general accuracy, and 3, recognition 0! 
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TABLE VI 
FACTOR LOADINGS 


Variables 


I 

1 General Accuracy -.....-._....... 945 
) Probably True and Probably False -- _.178 

Insufficient Data ~------------------ 869 
4 True—False ---------------------~- 362 
5 Omissions ~------------------------ sane, TS 
6 Caution _----~-—----- ----~—-------~. 210 
eo ee ee —.837 
ee error en sereeeny ere —.603 


statements not supported by the available 
evidence. Large negative loadings in the first 
factor are found for variable 7, going beyond 
the facts, and for 8, crude errors. 


The second centroid factor has a 0.707 load- 
ing for variable 4, accuracy with true and 
false statements, and a —o0.716 for 6, over- 
cautiousness. These two are logically oppo- 
sites since 4 is the proportion of true and 
false statements correctly marked, while 6 is 
a measure of the tendency to judge state- 
ments with less certainty than the data 
warrant. 


It may be noted that variables 1, 3, 7, and 
§ which have high loadings for the first fac- 
tor, have rather inconsequential loadings for 
the second. The two variables with high load- 


' ings in the second factor have low loadings on 


the first factor. None of the third and fourth 
factor loadings appear to be of great impor- 
tance. The dropping of the third and fourth 
factors does not leave out much common in- 
formation, as evidenced by the relative mag- 
nitudes of the estimated communalities which 
indicate the amount of variance taken into 
account for two and for four factors. For 
the purposes of this study rotation of the 
axes seems unnecessary. 


Omitting from the discussion variable 5, 
missions, only variable 2, accuracy with 
probably true and probably false statements, 
appears to have much information not in- 
cluded in either of the first two factors. The 
estimates of reliability of this variable are 
0.51 and 0.67, while the communality is only 
0.172, 

It might be feasible then for some purposes 
to obtain as a minimum only three or four 
scores for this test, rather than eight, with- 
out losing much of the information of the 





Estimate of 


Factor Loadings Communality 
II Ill IV Rien I+ TI+TII-tv 
207 —.198 —.131 .936 .992 
275 —.167 —.022 172 201 
—.387 —.177 —.083 .905 943 
707 —.155 —.192 .635 .692 
—.089 112 —.012 038 051 
—.716 171 —.299 557 675 
443 167 152 897 .948 
—.255 —.360 272 429 .632 


entire test. These might be variable 1, gen- 
eral accuracy, or variable 3, recognition of 
Statements not supported by the evidence; 
and either variable 4, true and false state- 
ments marked correctly, or variable 6, the 
tendency to be cautious in judging statements 
with less positivism than the data warrant. 
The third variable to be included if one were 
interested in shorter scoring is variable 2, 
general accuracy with probably true and 
probably false statements. 


CONCLUSIONS 


Internal evidence in this test indicates that 
it measures various skills in interpreting 
data. By definition there are five types of 
statements and the test gives an indication 
of the students’ skill in recognizing these 
statements. The test differentiates generally 
high from generally low ability and gives as 
well an indication as to the kind or type of 
ability. 

On the basis of the interrelations of the 
test variables, it may be concluded that 
possibly three of the variables reflect prac- 
tically all of the information given by the 
whole test. Considering the statistical find- 
ings, and the advisory use of the test data, 
these three variables which appear to be very 
important are: 


(3) Recognition of statements not sup- 
ported by the evidence. 


(6) Cautiousness—the tendency to be 
overcautious in judging the statements 
with less certainty than the data 
warrant. 

(2) Accuracy in judging the probably true 
and probably false statements. 
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III. COLLEGE STUDENTS AND 
CONTEMPORARY AFFAIRS 


O. O. ROYER 


Three basic assumptions are involved when 
a college utilizes contemporary affairs tests 
in its testing program. First, the college be- 
lieves that a knowledge of contemporary af- 
fairs is significant in the educative process of 
its students. The belief is general that the 
better educated person is one who knows a 
considerable amount about his contemporary 
culture, and the college shares that belief. 
Second, the school believes it can measure 
successfully the knowledge which students 
have of contemporary affairs. This factor in- 
volves the ability of the test makers to de- 
velop tests which are valid, reliable, prac- 
ticable, etc. It is quite conceivable that five 
different individuals could organize tests of 
current happenings on which each of the 
other five could fail to make a relatively high 
score. Third, the school believes that it can 
do something about the results of the tests 
in terms of pupil growth. No great purpose 
would be served for a school to learn how its 
students rated with respect to each other, or 
to other similar groups, unless that school 
also believes it can do something with its stu- 
dents to increase their knowledge of con- 
temporary affairs. Then it must also follow 
that the school will do something about the 
results of the tests. 

This study is concerned with the group of 
462 Education freshmen in terms of the three 





[Vol. Ps Ne 


basic assumptions which have been outlined 
It seems appropriate to discuss the second as. 
sumption first, namely, a school believes j 
can measure adequately its students know. 
edge of contemporary affairs. Table | 
presents a brief picture of the test which 
the College of Education used. Assuming 
that the test covers the major areas of cop. 
temporary affairs, it may be noted that each 
area is not represented by the same numbe- 
of items. Probably the last three areas 
Part I, with five, eight and twelve question: 
respectively, lack much in reliability. Th, 
low mean scores of these items relative + 
their standard deviations indicate the: 
distributions are skewed. 

There was no statistical study made y 
respect to the reliability and the validity 
this test as a measure of knowledge of cop- 
temporary affairs, but we shall assume in 
rest of this discussion that the test 
reasonably reliable and valid. 

Table II indicates positive relationshi 
among all of the various areas in Part I. Thy 
knowledge of national personalities is partic:- 
larly associated with knowledge of intern: 
tional personalities. An understanding of rea- 
sons for national events is closely associated 
with the students’ abilities to understand re:- 
sons for international events. There appear: 
to be a rather high degree of relationshi 
between knowledge of personalities an! 
events, both national and international. One 
might have expected, however, that “Word: 
in the News” would have had a higher rela- 


TABLE I 


MEANS AND STANDARD DEVIATIONS OF SUB-SCORES ON COOPERATIVE CONTEMPORARY AFFAIRS TEst 


Parts of Test 


ae ea ee TE ree 


Part I—Political-Social ._..........._____.__-_ 
Identification of Personalities—National —___ 


Identification of Personalities—International 
Knowledge of Events 





Science 
Reasons for Events—National 
Reasons for Events—International 
Part II—Contemporary Culture 
Books 
Stage 
Music and Radio 
Movies 





OS eee 
Knowledge of Events—International _______-_ 
Meaning of Words in the News _____--__-__ 


Ohio State 

Highest Education Freshmen 

Possible Standard 

Score Mean Score Deviation 
sccicencaaicin 300 71.36 30.22 
rinsieahienceehinnatits 150 29.80 16.87 
caciareesnpaamsaiye 22 8.02 3.35 
a re 18 ake 2.89 
Seiad 30 10.66 Beil 
eer ee eee 40 12.51 6.02 
eae Se ee 15 4.19 3.1 
ieaiisiiiiads 5 50 83 
sasigilaiaalcana s .95 1.85 
RT 12 1.11 13 
ee ae 150 41.09 18.24 
ouiiciiaeblicistins 50 14.27 0.68 
siechcniiaaieaaciin 17 4.81 3.08 
A ee ee 22 8.83 3.30 
al A 43 25.81 8.80 


18 2.76 2.33 
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TABLE II 


INTERCORRELATIONS OF THE SUBDIVISIONS OF THE COOPERATIVE CONTEMPORARY 
AFFAIRS TEST (PART I) 


N = 462 
1. Total (Past 5) ......... 
> Identification of National Personalities _ 


9 Identification of International Personalities __- 


4 Knowledge of National Events 


5, Knowledge of International Events ---_-~-~~ 


: Meaning of Words in the News 
7, Science 


_ Reasons for National Events ~_--------_--~_~- 


9 Reasons for International Events 


tionship with other parts of the test. The 
comparatively low correlations may be due 
to an insufficient number of items in this part 
of the test. 

In Table III we see a close inter-relation- 
ship between the various tactors which make 
up the “Contemporary Culture” aspect of the 
whole test. The correlation between a know- 
ledge of books and of movies is significant 
by its being lower than the other correlations. 
\n important point is indicated for parents 
and educators. Some young people may be 
spending too much of their time with books. 
Part of that time might well be shared with 
other forms of amusement. On the other 
hand, many youth may be devoting too much 


time to the movies. These youth might well 
profit by more reading. 


TABLE 
INTERCORRELATIONS OF THE SUB-TESTS 
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The first basic assumption discussed in this 
study stated that a school believes that a 
knowledge of contemporary affairs is signifi- 
cant in the educative process of its students. 
In considering this phase of our problem, we 
refer to Table IV which shows the relation- 
ship of the total scores of the Contemporary 
Affairs Test with rank in class, psychological 
test score, autumn point-hour-ratio, and Sur- 
vey of Education course grades. The rank in 
class refers to the students’ rank in their high 
school graduating classes, 2 indicating upper 
third, 1 middle third, and o lower third of 
the class. Form 19 of the Ohio State Psycho- 
logical Test was the basis for the second 
variable. The autumn point-hour-ratio was 
the third variable. This is the customary av- 
erage of University grades whereby four 


4 


III 


OF THE COOPERATIVE CONTEMPORARY 


AFFAIRS TEST (PART II) 


N= 462 
Total (Part IT) 


TABLE 


Music & 
Radio 
57 
45 
52 


38 
31 


Books 
58 


55 
A5 
26 
35 


Art 
43 
35 
36 
31 
38 


Movies 
Pe i | 
26 
44 
38 


38 


Stage 
.66 
.55 


52 


44 
36 


IV 


INTERCORRELATIONS OF TOTAL AND Part TOTAL Scores OF THE COOPERATIVE CONTEMPORARY 
AFFAIRS TEST AND Four COMMON VARIABLES 


N = 462 
Total Score 
Seore (Part I) 
Score (Part IT) 
Rank in Class 
Psychological Test 
Autumn P.H.R. 
Survey Grade 


Total PartI 


Rankin Psych. Autumn Survey 
Class Test P.H.R. Grade 
18 
14 
19 


45 
44 
37 


Part IT 


82 
48 
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points are awarded for each hour of A, three 
points for each hour of B, two points for each 
hour of C, and one point for each hour of D. 
The grade E (failure) receives no points. The 
Survey of Education grade was a grade for 
a particular course, especially designed for 
Education freshmen for purposes of helping 
them to adjust themselves to college work 
in terms of their chief interests and aptitudes. 

Now it would seem that one’s rank in a 
high school graduating class would indicate 
to a fair degree a particular student’s edu- 
cation in comparison with his fellow students. 
If our basic assumption is valid, one would 
expect a fair correlation between the rank 
in class and knowledge of contemporary af- 
fairs. The correlation of 0.18 does not bear 
out that expectation. Thus, if we hold to our 
basic assumption, one of two things seems to 
be true. Either the Contemporary Affairs 
Test is not valid, or the high schools use poor 
methods of measuring the amount of educa- 
tion which their students possess. A combina- 
tion of these two interpretations is also 
possible. 

The Ohio State Psychological Test was ad- 
ministered in the first week of school before 
the students were greatly influenced by uni- 
versity life. One is not surprised, however, 
at the high degree of relationship existing be- 
tween the Psychological Test and the Con- 
temporary Affairs Test as shown by a cor- 
relation of o.41. In brief, one can say that 
a group which does well on the Psychological 
Test tends to do well on the Contemporary 
Affairs Test. 

rhe correlation between the autumn point- 
hour-ratio and the Test total is 0.32. This 
would indicate that a higher degree of rela- 
tionship exists between college grading and 
knowledge of contemporary affairs than 
exists between high school grading and 
knowledge of contemporary affairs. 

Of the four outside variables studied, then, 
it is apparent that the Psychological Test 
shows the highest relationship with the total 
score of the Contemporary Affairs Test, and 
also with each of its two parts. Thus, one can 
very well say that the group which makes 
high scores on the Psychological Test also 
tends to have a fair degree of knowledge 
about contemporary affairs, both political- 
social and cultural in nature. 

The third basic assumption was that a 
school believes it can do something about in- 
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creasing its students’ knowledge of cop. 
temporary affairs. This involves more thay 
just presenting knowledge to be learned, fo; 
knowledge learned today will not be consid. 
ered contemporary knowledge one year from 
today. Thus, the problem at once become 
complicated and the school must of neces. 
sity develop in its students the desire to 
acquire knowledge of contemporary culture 
as it is developing from day to day. A prac- 
tical question immediately arises. To what 
extent should a school expect its students to 
develop this desire for the acquirement oj 
contemporary knowledge? Since the desire 
itself means nothing unless it results in the 
actual acquiring of knowledge, the school has 
two things to do in order to establish a basis 
for the answer to the above question. First, 
it must secure an accurate picture of its own 
students’ accomplishments. Second, it should 
compare its group with other similar groups 
who have been measured with similar testing 
devices. 

In this connection, it is possible to com- 
pare the Ohio State freshmen with a great 
number of sophomores from sixty-six colleges 
where the same Contemporary Affairs Test 
was used.’ If we recognize the fact that a 
difference of at least one year in training 
existed between the two groups, we are ap- 
parently justified in believing that the Ohio 
State group did well in this comparison as is 
indicated by Table V. 

Another means of comparison, which als 
should be a measure of growth, will materia- 
ize when the group of 462 return to the Ohi 
State University as sophomores, as it is the 
aim of the College of Education to again ad: 
minister to them a Contemporary flair 
Test. 

If as sophomores, the Ohio State group 
make scores similar to a general college group 
of sophomores, another point should be 
clarified before an interpretation is attemptec 
If the colleges in the general group are mak- 
ing definite attempts to assist their students 
to acquire knowledge of their contemporary 
lives, then the Ohio State University College 
of Education may have the right to a feeling 
of satisfaction. If, however, the figures tor 
the general group would be the result of the 
promiscuous use of tests by the colleges 
the group, the College of Education may wel 
be disappointed with its efforts. With ™ 


1 Eckert, Ruth E. “Who Are the ‘Cultured’ in Our Col 
leges?”” The Educational Record, January, 1939. 
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TABLE V 


COMPARISON OF SOPHOMORES OF SIxTy-Six COLLEGES WITH THE OHIO STATE 
UNIVERSITY EDUCATION FRESHMEN 


Ohio State Freshmen 


Part I 
Mean Score -~------------------------ 29.8 
Standard Deviation —.—-------.-.------ 16.9 
Number .------- ~—-—---~—----~-~--------- 462 


training, the Ohio State group should measure 
up to a general group. With training, they 
should exceed the general group. 


SUMMARY 


For a college or university to administer 
intelligently a contemporary affairs test as a 
part of its testing program, it must accept 
three basic assumptions. First, the ade- 
quately educated individual is one who knows 
considerable about contemporary affairs. Sec- 
ond, the school believes it can measure its 
students’ knowledge of contemporary affairs. 
fhird, the college or university believes it can 
influence its students in such a way that they 
will acquire knowledge of contemporary 
affairs as the new knowledge develops. 

The Ohio State University administered 
the Cooperative Contemporary Affairs Test 
to its Education freshmen in the Autumn 
Quarter of 1938. By the very act of ad- 
ministering this test, the University gave evi- 
dence of accepting the first assumption. The 
results indicate, however, that knowledge of 
contemporary affairs is not closely associated 
with other factors which are commonly indi- 
cative of student education, namely, grades 
and rank in class. It is of interest to note 
that there is a high degree of relationship be- 
tween the test results and the Psychological 
Test, which indicates that those with superior 
ability tend to know more about their sur- 
rounding culture than those of lesser ability. 

By giving the Contemporary Affairs Test, 
the College of Education indicated its ac- 
ceptance of the second basic assumption. 
However, the items in some parts of the test 
are so few, the mean scores are so low, and 
the intercorrelations are of such a degree that 
one immediately is assailed by serious doubts 
about the validity of the test which was used. 
Also, there is little evidence that the test is 
reliable. 

In formulating the purposes of the Survey 
course, the College of Education also gave 
evidence that it accepted the third basic as- 





College Sophomores 


PartII Total Part I PartII Total 
41.1 71.4 38.4 45.4 83.7 
18.2 30.2 22.9 21.7 39.3 
462 462 6378 6378 6378 


sumption. However, no evidence has yet been 
presented that this group of 462 students im- 
proved in their knowledge of contemporary 
affairs, as no comparable test was given at 
the end of the year. Further testing is a much 
needed next step. Definite procedures should 
be established not only for the teaching of 
a knowledge of contemporary affairs, but also 
for the furthering of the desire in the indi- 
vidual to master new knowledge as it appears. 


IV. LIBERALISM AND CONSISTENCY: 
A STUDY OF SOCIAL ATTITUDES 


C. C. Grepons AND W. A. B. SCHRADER 


The results obtained by giving a test called 
“A Scale of Beliefs,’ to a group of education 
freshmen constitute the data for this study. 


The test consists of one hundred eighty-six 
statements which deal with the following 
types of social problems: 


(1) Democracy 

(2) Economic Relations 
(3) Labor 

(4) Race 

(5) Nationalism 

(6) Militarism 


The students were toid that “there are no 
right or wrong answers; you are to express 
your point of view about these statements.” 
The students indicated their attitude toward 
each statement in the test by marking them 
as either “Agree”, “Disagree”, or ‘“Uncer- 
tain”. 

The following statements are indicative of 
the types of statements included in the test: 


1. In a democracy, it is often necessary to 
give up one’s own desires or to change 
one’s actions so as to benefit others. 
(Democracy ) 


1“A Scale of Beliefs’, Progressive Education Association 
Tests 4.2 and 4.3, Evaluation in the Eight-Year Study. 
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2. Our economic progress would be greater 
and our standard of living higher if our 
industries were publicly owned and op- 
erated. (Economic) 

3. The government should provide for the 
unemployed through taxation. (Labor) 

4. All positions in the political and eco- 
nomic world should be open to any man 
with the ability to fill them, regardless 
of race. (Race) 

5. Other countries excel us in many 
respects, just as we excel them in many 
respects. (Nationalism) 

6. A man who is sincerely against war 
should not be made to go to war. 
(Militarism) 


If a student agrees with one of the fore- 
going statements, he receives one score to- 
ward liberalism in the area indicated. If, on 
the other hand, he disagrees with one of these 
statements, his response is counted as “con- 
servative’’. In this way, the test papers were 
scored for liberalism and conservatism in 
each of the six areas mentioned above. The 
number of items which a person marks “un- 
certain” is taken as his uncertainty score. One 
week after the students had completed one 
part of the test, which consists of ninety-three 
items, they were given a second part which 
contains items similar to those in the first 
part. Each statement in part two expresses a 
viewpoint directly opposed to the viewpoint 
expressed by the corresponding statement in 
part one. By considering the answers to these 
pairs of statements a consistency score was 
derived for each of the areas of thought in- 
volved. The six statements from part two 
which correspond to the six statements listed 
above from part one are: 


1. In a democracy, everyone should be 
free to do as he pleases and should not 
be forced by the government to do 
things he does not wish to do. 
( Democracy) 

. Production based on the system of pri- 
vate ownership is essential for economic 
progress. (Economic relations) 

3. The government has no right to tax the 
public generally to provide for those 
who are out of work. (Labor) 

4. Negroes should not be allowed to fill 
positions involving leadership over white 
people. (Race) 


tN 





5. The United States is superior to mos 
other nations in such important respects 
as government, educational opportuni. 
ties, family life, and morals. (National. 
ism) 

6. No matter what his personal beliefs re. 
garding war, no man has the right ty 
refuse to serve his country in times o 
war. (Militarism) 


It is apparent in most cases that a person 
who agrees with a certain statement in part 
one will, if he is consistent, disagree with the 
corresponding statement in part two. Some 
of the statements in the first ninety-three are 
so stated that the “liberal” will agree with 
them, and some are so stated that the 
“liberal” will disagree with them. 

The test yields twenty-eight scores ey- 
pressed in percentages as follows: Scores for 
liberalism, conservatism, uncertainty, and 
consistency, for each of the six areas tested, 
and total scores for liberalism, conservatism, 
uncertainty and consistency. In addition + 
these twenty-eight test scores, sex, race, in- 
telligence test score, and college grades are 
included in this study. 


Standard Hollerith equipment and methods 
were used throughout this study in the 
computation of the means, standard devia- 
tions, and coefficients of correlation. The 
original population included both negro and 
white, male and female students. Means and 
standard deviations were computed for the 
four race-sex groupings, in each of the vari- 
ables. At this point the negroes were elimi- 
nated in order to obtain a more homogeneous 
population for the calculation of intercorrela- 
tions. The negroes were not studied sepa: 
rately because the size (N = 25) and rep- 
resentativeness of the sample did not seem 
to justify such an analysis. It may be noted, 
as indicative of the validity of the test, that 
the negroes were decidedly more “liberal 
and more consistent than the whites in at- 
titude toward race problems. For the remain- 
ing four hundred thirty-seven subjects, 
means, standard deviations, and correlation 
coefficients were obtained for the following 
variables: 


Variable number— 


1. Scholastic Grades? 
2. Intelligence 
as The grades used were point-hour ratios for autumn quarter 
38 
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Liberalism in: 

;. Democracy 

4. Economic Relations 

5. Labor 

6. Race 

>. Nationalism 

8. Militarism 


Consistency in: 

Democracy 

:o. Economic Relations 
Labor 

12. Race 

12. Nationalism 

14. Militarism 

is. Total Liberalism 

6. Total Conservatism 
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Unfortunately, these tests were scored in 
such a way that it was not feasible to cal- 
culate coefficients of reliability for the test. 
The Progressive Education Association, pro- 
ducers of the tests, report reliabilit, coeffi- 
cients which are based on 500 subjects, rang- 
ing from grades 9 to 12. Reliability coeffi- 
cients were estimated from the data of the 
present study by the method of Kuder and 
Richardson.® These two sets of reliability 
coefficients are presented in Table II. 

The relation of the several liberalism 
scores to intelligence and college grades is 
shown in Table III. 


Comparisons between means for different 
kinds of liberalism would not be valid be- 


. r . 3 ; i c de ( M. ‘4 chardson, . » Orv ; e 

I7- i tal L ncertainty mB gf eiebaine”™ ge LS 1 Tsiiate 
18. Total Consistency September, 1937. 

TABLE I 
Means, STANDARD DEVIATIONS, AND CORRELATIONS FOR LIBERALISM VARIABLES 

Stand- 

Ec. ard De- 

Democ. Rel. Labor Race Nation. Milit. Total Mean viation 

ic} sco ginasaeaateek bei ik oo 1 06 ol 48 .76 65.6 10.0 

, M 46 52 30 40 46 58 61.9 9.1 

nic Relations ___-- F D3 8 22 38 .40 .70 52.2 18.4 

M 46 55 20 85 83 G4 49.5 18.1 

aa ol 48 43 40 ot 71 65.7 14.7 

M 52 55 30 36 se 60 63.6 14.2 

ee 36 nae 43 46 26 56 59.1 23.1 

M 30 .20 30 38 33 58 58.5 21.3 

1 eee ee F 51 oO 40 46 54 .70 61.6 15.7 

M 40 35 26 oS 56 62 56.4 15.6 

NE es F .48 .40 2 26 04 67 62.3 13.7 

M 4 33 mf 83 6 61 57.5 14.0 

a F .76 70 71 56 .70 67 60.2 10.9 

M 08 64 .60 58 62 61 57.7 10.4 


LIBERALISM 


{he correlations between liberalism in 
(ifferent areas, computed separately for men 
and women, are shown in Table I. For all 
the tables, the figures for men are in italics 
ind those for women are in arabic type. The 
hgures are also labelled ““M” and “F”, 


TABLE II 
\ELIABILITY COEFFICIENTS FOR LIBERALISM 
VARIABLES 
Liberalism in: P.F.A. K-R Method 
F M 
(3) Democracy .______ 80 58 46 
\4) Economie Relations .86 75 74 
(5) pS ine Ee 719 71 .68 
2 _ Fee 87 81 77 
(') Nationalism _____ .70 63 60 
(8) Militarism ___.___ . 82 71 71 


TABLE III 


CORRELATIONS BETWEEN LIBERALISM, INTEL- 
LIGENCE, AND COLLEGE GRADES 


Liberalism in: Correlation with: 


Intelligence Grades 

(3) Democracy ____ F 30 34 
M 25 06 

(4) Economie Rela- 
tems ..... F 14 .20 
M 08 06 
(5) Laber ........ F .29 25 
M 12 07 
(6) Meee ......... F aa 24 
M 18 08 
(7) Nationalism ___ F 30 31 
M 35 42 
(8) Militarism ____ F 3 36 
M 28 .27 
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cause they would reflect nothing more than 
the difficulties of the different tests. Sex 
differences in means, however, may be legit- 
imately compared. We shall consider first the 
question of sex difference in means of the 
liberalism variables. The only differences 
that are presented in Table IV are those that 
are at least twice their standard errors. For 
every difference included in Table IV the 
mean for the women was larger. 


TABLE IV 


CRITICAL RATIOS FOR SEX DIFFERENCES 


IN MEANS 
Name of Critical Ratio 
Variable for Sexes 
Intelligence : : 3.92 
Liberalism in Democracy ~---~~- 3.93 
Liberalism in Nationalism —__-~~ 3.29 
Liberalism in Militarism —-~~-~- 3.39 


For determining the significance of sex 
differences in the intercorrelations, these cor- 
relations were changed into Fisher’s “z” 
function.* The correlations for which there 
are significant sex differences are those be- 
tween liberalism in democracy and (1) total 
liberalism (3.10), and (2) college grades 
(2.91). These numbers are the critical ratios 
which indicate the extent to which the cor- 
relations were higher for women. 

In addition to the sex differences, the dif- 
ferences between the magnitudes of certain 
correlations are interesting. For instance, 
Table III shows that for men the correlation 
between intelligence and_ liberalism in 
economic relations is 0.08 whereas the cor- 
relation between intelligence and liberalism in 
nationalism is 0.35. How much emphasis 
should be placed on such a difference? This 
paper presents one hundred twenty-nine in- 
tercorrelations for women and the same num- 
ber for men. It would be possible to make 
8256 comparisons like the one above for men 
and the same number for women. From 
Tables V and VI the reader can estimate the 
significance of the difference between any 
correlations which he wishes to compare in 
this study. Tables V and VI show for the 
two groups in this study the critical ratios of 
the differences in every case where the 
critical ratio is larger than 2.00. 

Table IV shows that one of the largest sex 
differences in means is in intelligence. As ex- 


“Fisher, R. A. Statistical Methods for Research Workers. 
Oliver and Boyd, London 1938 z == % [log, (1 + r) — log, 


(1—r)]. 
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perience has shown to be true of educatis- 
students, the women in this study haye - 
higher mean intelligence test score than th: 


men. The question immediately arises 4s ; 


TABLE V 


CRITICAL RATIOS OF THE DIFFERENCES Betwpp: 
r, AND r; WHEN N = 288 (e£4., tH; 
FEMALE POPULATION IN THIS StuUpy) 


Yr: 


30 .40 .50 60 #7 

10 2.538 3.95 5.48 7.14 99 

.20 2.67 4.23 5.88 72 

r; 30 2.86 4.55 64 
40 3.23 57 

50 7 


TABLE VI 


CRITICAL RATIOS OF THE DIFFERENCES Betwe; 
ri, AND rz WHEN N = 149 (k.4.. tH: 
MALE POPULATION IN THIS StTuUpy) 


.40 50 = .60 7 
10 2.83 3.92 5.21 
.20 3.03 £26 (fj 
a 30 2.08 3.30 4.71 
"40 233 37 


50 2.7 


whether this difference may not explain th 
other sex differences presented in Table |' 
The correlations in Table III between int 
ligence and different kinds of liberalism 
so low, however, as to indicate that int 
ligence does not bear enough relationship 
liberalism to cause any marked sex differ 
ences. 


Table I shows considerable interrelation be- 


tween different kinds of liberalism. Ho 
much of this interrelationship is due to th: 
common factor of intelligence? Table VI 
shows the correlations of Table I after the 
linear influence of intelligence has been re 
moved by partial correlation. 

It is noteworthy that the correlations © 
Table VII are still significantly greater than 
zero after intelligence has been held constant 
The largest reduction caused by partialling 
out intelligence was 0.08 for women and 0.0 
for men. The different liberalism scores see” 
to have something in common besides inte- 
ligence. It must be recognized that the on) 
liberal behavior involved here is in making : 
certain response to a written statement. 

Can the intercorrelations between different 
kinds of liberalism be taken as an indication 
of a liberalism factor? Part of the argv: 
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TABLE VII 


RRELATIONS BETWEEN LIBERALISM VARIABLES WITH INTELLIGENCE HELD CONSTANT 


Democ. 


waa [eae 
eats M 
mic Relations . ¥ 52 
M 46 
F AT 
M 51 
F wool 
M £7 
desi ae |: 47 
M PA 
vl F 4 
M 42 
here must be based on Table VIII, 


shows correlations between different 
iberalism scores and the corresponding con- 
stency scores. Table I shows correlations 
between different kinds of liberalism. The 
means’ of these correlations in Table I are 
123 for women and 0.388 for men. One 
cht regard these figures as evidence for a 
eralism factor. However, these means are 
t little larger than the means of the cor- 
relations in Table VIII between different 
nds of liberalisms and the corresponding 
sistencies. The means in the latter case 
ire 0.334 for women and 0.321 for men. 
The correlations on the diagonal in the 
wer part of Table VIII are noticeably high; 
e., liberalism in race with consistency in 
e, etc. 


I s were computed before the correlations were 
two decimal places. 


203 
Ec. Rel. Labor Race Nation. Milit. 
By AT wl AT 41 
46 1 £7 34 42 
AT 19 256 38 
5 19 Pf 2 
AT 08 34 29 
55 28 a4 1 
19 08 42 18 
19 28 me 30 
36 me? 42 .48 
35 Jt BP 51 
38 29 18 18 
2 31 30 1 


An interesting sex difference occurs in that 
the correlation between liberalism in race and 
consistency in race is larger for women than 
for men. The critical ratio is 2.09. A differ- 
ence that is hard to explain occurs in the 
correlation between total liberalism and con- 
sistency in attitude toward labor. The cor- 
relation for men is higher to the extent of a 
critical ratio of 2.38. 


CONSISTENCY 


On the first form of the test, ninety-three 
statements are presented, some worded so 
that a liberal answer is “Agree” and some 
worded so that a liberal answer is “Disagree”’. 
On the second form of the test, each state- 
ment is reworded so that it expresses the op- 
posite point of view. Consistency is measured 
by the extent to which the person makes the 


TABLE VIII 


CORRELATIONS BETWEEN LIBERALISM AND CONSISTENCY SCORES 


stency in: Democ. Ec. Rel. 

ntelligence ic sete) 27 24 
M 27 85 
OR oe ee igale F 34 .24 
M 19 34 

alism in: 
MOCTROY on ee F 59 .20 
- M 55 27 
Economie Relations _. F 36 31 
. M 26 30 
Mee. ooo F .39 .22 
. M Ab 16 
ee .29 14 
; M 25 26 
Nationalism __..__ | F 41 28 
M 43 41 
Militarism _...___ | F .34 .29 
M 25 36 
Total Liberalism _____ F .53 .30 
M 46 44 


Total 
Labor Race Nation. Milit. Consist. 

wl 20 .29 36 .oD 
24 mF 28 26 39 
33 21 35 40 44 
.20 08 8&2 24 35 
.34 Rs - 30 36 AT 
44 By 35 23 AL 
.36 13 3 3 38 
43 17 22 25 Bg 
51 25 “oo wl 46 
58 03 .26 19 39 
.28 62 .25 .20 42 
39 47 25 82 42 
.04 .26 59 41 52 
37 24 46 40 52 
27 19 .43 .65 45 
39 20 .26 55 47 
44 38 48 51 59 


-61 33 39 51 -61 
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TABLE IX 


MEANS, STANDARD DEVIATIONS, AND CORRELATIONS FOR CONSISTENCY VARIABLES 


Ee. 
Democ. Rel. Labor 
Democracy I 20 45 
M 38 49 
Economic Relations. F .30 oo 
M 38 Al 
Labo: k 45 oo 
M 49 41 
Race I .24 me .28 
M ye 26 SL} 
Nationalism F 36 23 Dy | 
M 38 a? 36 
Militarism I 45 By -< | 38 
M m+ 43 42 
Total F 68 Lol .63 
M 66 70 69 


same types of responses (i. e., either liberal 
or conservative) on the corresponding state- 
ments of the two test forms. Table [LX shows 
the correlations between different consistency 
scores. 

The reliability coefficients of the consist- 
ency scales were not directly obtained. The 
reliabilities were estimated by the Kuder— 
Richardson method referred to above. The 
Progressive Education Association reports re- 
liability coefficients for these scales, based on 
500 subjects in grades 9 to 12. Both sets of 
reliability coefficients are presented in 
Table X. 

The sex differences shown in Table IX 
were tested for significance. The variables for 
which the differences in means are significant 


TABLE X 


RELIABILITY COEFFICIENTS FOR CONSIST- 
ENCY VARIABLES 


Seale Coefficient of Reliability 

Consistency in: P.E.A. K—R Method 
F M 

(9) Democracy ____--~ .68 Al 39 
(10) Economic Relations .88 35 40 
eee Se 61 46 39 
C32) O08 WW nnn .63 51 46 
(13) Nationalism ___-~ .66 .39 25 
(14) Militarism ____-- .64 56 45 


are consistency in democracy (2.93) and 
consistency in economic relations (2.03). 
These figures are the critical ratios. Both 
means are larger for the women than for the 
men. To facilitate their comparison the cor- 
relations were changed to Fisher’s “z” func- 
tion. The only correlation for which there is 
a significant sex difference is that between 


Stand 


Standard 
Race Nation. Milit. Total Mean Deviatio; 


24 206 45 68 60.2 12 
19 38 32 66 56.5 12 
Pyar 22 | 50 59.4 16. 
26 37 43 70 55.8 174 
.28 ot 38 63 63.6 15.5 
33 36 42 69 62.6 14.8 
.24 19 A9 61.2 21. 
15 wh 49 57.5 20.4, 
24 46 4 56.3 178 
15 43 o9 55.4 f 
19 46 64 60.1 16.0 
3h 43 67 57.8 
49 54 64 59.4 9 
49 59 67 56.4 


consistency in economic relations and total 
consistency. The correlation for men 
higher to the extent of a critical ratio of 3.14 
This is probably an underestimate because 
of the correlation of the two variables. 


Table IX shows some correlation between 
consistencies in different areas of the test. T 
determine how much of this correlation was 
due to the common factor of intelligence 
Table XI was calculated to show the relation 
between different consistency scores when in- 
telligence is held constant. The correlations 
in Table XI are still significantly greater than 
zero after intelligence has been held con- 
stant. The largest reduction resulting from 
partialling out the intelligence test score is 
0.08 for women and 0.07 for men. 

The mean of all the correlations in Tab’ 
IX between different consistency scores is 
0.317 for women and 0.249 for men.° These 
figures probably cannot be taken as evidence 
for a general consistency factor, however, be- 
cause the means of the correlations between 
liberalism scores and consistency scores are 
0.334 for women and 0.321 for men. (S¢€ 
Table VITT) 


CONSERVATISM AND UNCERTAINT! 


As explained at the beginning, there wer 
conservatism and uncertainty scores in eact 
of the six areas as well as liberalism and con- 
sistency scores. This study has been limite 
chiefly to a consideration of liberalism an: 
consistency. Total conservatism and uncer 
tainty scores were studied, however, in rele 
tion to intelligence, grades, and total libera- 


*See footnote 5. 
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TABLE XI 
CORRELATIONS BETWEEN DIFFERENT CONSISTENCY Scores WITH INTELLIGENCE HELD CONSTANT 
Democ. Ec. Rel. Labor Race Nation. Milit. 
ical cola ceases ee F 26 40 19 30 39 
: : M 32 45 16 33 .27 
Economie Relations ------~------- F .26 .27 18 16 20 
M 32 36 23 30 387 
ee ee 40 27 22 31 30 
M Ad 36 31 31 38 
VN Cee ae a F 19 18 22 19 12 
M 16 23 31 Bi i 32 
' NetID oo. iin ccincinnnmmne F .30 .16 ol 19 09 
9 M 33 30 31 11 39 
IR acc ites actin alana F 09 .20 .30 12 3) 
M 27 37 38 32 39 
TABLE XII 
MEANS, STANDARD DEVIATIONS, AND CORRELATIONS FOR INTELLIGENCE, 
GRADES, AND TOTAL SCORES 
Total Total Total Total 
tg Liber- Conserv- Uncer- Consist- Standard 
use Intell. Grades alism atism tainty ency Mean Deviation 
iifieeees ——..-.- F 60 Si 60Ul eK .8 35 65.1 24.8 
an M 58 21 —.40 06 39 55.0 26.1 
= Reali, << cano F 60 37 —27 —.13 A4 2.39 15 
7 M 58 17 —9 32 35 2.28 67 
was Total Liberalism ___ F wT ot —.27 —.69 59 60.2 10.9 
nce. M 21 17 —.30 —.50 61 57.7 10.4 
tion Total Conservatism. F —.21 —.27 —.27 —12 —.35 20.5 9.6 
i M —40 —.29 —.30 — 33 —.28 22.5 9.4 
eal Total Uncertainty.. F —13 —13 —69 —.12 —.27 20.0 10.5 
ions M .06 32 —.50 —.3$ —.29 21.1 13.1 
than Total Consistency __ F B85 44 59 —.35 —.27 59.4 10.9 
a M 39 35 61 —.28 —.29 56.4 10.7 
from 
fe ism and consistency scores. These results are grades (4.45), total liberalism (2.89) and to- 
presented in Table XII. tal conservatism (2.21). These numbers are 
- In Table XIII, the coefficients of reliability the critical ratios. 
: which the Progressive Education Association 
hese a ith SUMMARY 
a present for the total scores are compared wit ; 
a those estimated by the Kuder—Richardson The Education freshmen at Ohio State uni- 
jal method. versity were given an attitude test to deter- 
wee! : , , : * St EE EY Oe ee » Rhie 
a Some of the sex differences presented in gare reactions to orga oA the bag 
(See Table XII are significant. For total liberal- © emocracy, economic relations, labor, 
sm and total consistency, the women have ‘ace, nationalism and _ militarism. These 
higher scores to the extent of critical ratios of | tests were scored for liberalism, conservatism, 
230 and 2.77 respectively. For total con- uncertainty and consistency in each of the 
rvatism and total uncertainty the differ- six fields. Intelligence test score and first 
were ences are in favor of the men, the critical Taste XIII 
each ratios being 2.09 for conservatism and 0.91 
| con- lor uncertainty. As indicated previously, the RELIABILITY COEFFICIENTS FOR THE 
mited women scored higher on the intelligence test. TOTAL SCORES 
and The critical ratio of this difference is 3.82. P.E.A. K-R Method 
incer- A comparison of the correlations by Fish- : ‘ F M 
rela- e's “2” function shows that the only signifi- Total Liberalism -—____ 92 20 88 
peral- cant sex diff in Table XII It f Total Conservatism ___ .90 91 90 
“ex Cillerences in lable result Irom otal Uncertainty ____ .88 93 = 96 


the correlation of total uncertainty with: 





Total Consistency _____ .96 80 79 
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TABLE XIV 
MEANS OF DIFFERENT TYPES OF 
INTERCORRELATIONS 


Women Men 
15 r’s for Liberalism vs. Liber- 


alism 4233 3885 
36 r’s for Liberalism vs. Con- 

sistency . «000 3209 
15 r’s for Consistency vs. Con- 

sistency o1%3 3489 


quarter college grades were also included in 
the study. The means, standard deviations 
and intercorrelations were obtained for these 
variables for a group of four hundred thirty- 
seven men and women. All the sex differences 
found were tested for significance and the 
differences which are twice their standard 
errors are reported. For intelligence, grades, 
and the fourteen liberalism and consistency 
variables the means were all higher for women 
than for men. Eight of the critical ratios were 
above 2.00. Only for total conservatism and 
total uncertainty do the men have a higher 
mean. Of the one hundred twenty-nine dif- 
ferences in correlation, eight of them were 
more than twice their standard errors. 

The intercorrelations between different 
liberalism scores were significantly greater 
than zero. These correlations were not sub- 
stantially reduced when intelligence was held 
constant. The same may be said for the inter- 
correlations between different consistency 
scores. These intercorrelations raise the ques- 
tion as to whether there may not be a general 
liberalism and a general consistency factor. 
Such a contention is weakened, however, by 
the data of Table XIV which shows the 
means of the different types of intercorrela- 
tions. There seems to be a stronger argument 
for a liberalism factor than for a consistency 
factor. If the figures in Table XIV_ had 
been based on correlation coefficients that had 
been corrected for attenuation, the case for 
generality of the traits would look stronger. 
Perhaps the intercorrelations are large enough 
to justify one to speak of liberalism in A, 
liberalism in B, etc. If the traits were un- 
correlated, one would not be justified in using 
the term “liberalism” in every case. The case 
for general liberalism is not strong. Taking 
.423 from Table XIV as the average inter- 
correlation between different liberalism 
scores, one arrives at the interpretation that 
there are only 59 chances in 100 that a per- 
son above average on one kind of liberalism 
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will be above average on a second kind 
liberalism, while chance alone would give ;- 
chances in 100. 

Tables are given from which one may esti. 
mate the critical ratio of the differe: 
tween two correlations which he wishes ; 
compare. Besides: the intercorrelations 
tween liberalism scores and consiste 
scores in different areas of thought, the jr 
tercorrelations are given between intelligence 
grades, total liberalism, total conservatis; 
total uncertainty and total consistency score: 

It is important to notice how low the , 
sistency scores are in this study. The s 
dent was presented with two statements dy 
ing the course of the test. If he answered bo 
statements liberally, or conservatively, he was 
given credit for being consistent. The mea 
scores show that the responses were consistent 
less than sixty per cent of the time. hy 
chance alone the responses would have beer 
consistent thirty-three percent of the time 
A mean score of sixty per cent may be inter- 
preted to mean: (1) The students gay 
thought to their responses; hence this | 
is invalid; (2) The students fail to recogni: 
that the second statement in the pair is 
direct conflict with the first; (3) The low 
consistency scores indicate that freshmen ar 
confused about their opinions on_ social 
issues. There are several bits of evidence 
which support this third interpretation. The 
consistently positive correlations presented i 
this study would have been highly improba! 
if the students had simply guessed. Hig! 
consistency scores are associated with lov 
uncertainty scores. There seems to be som 
relationship between the amount of experienc: 
and discussion in a certain field and the con- 
sistency with which college freshmen answer 
questions in this field. Following are the si 
fields involved in this study ranked in the 
order of their mean consistency score: 


Women Men 
1. Labor 1. Labor 
2. Race 2. Militarism 
3. Democracy 3. Race 
4. Militarism 4. Democracy 
5. Economic Rela- 5. Economic Re!* 


tions tions 
6. Nationalism . Nationalism 


a 


The ranking is roughly the same when the 
fields are arranged in order of mean liber! 
ism score. Are not the fields here ranked al 
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cording to the amount of experience and V. ANALYSIS OF VARIANCE APPLIED 

discussion in which the average college fresh- TO LIBERALISM SCORES 
man has participated in connection with the w A hk Gee 

fields in question? 

The problem as to whether or not a sig- 

CONCLUSIONS nificant relationship exists between two vari- 

; ; ' ables, one of them quantitatively measured 

[he women included in this study are and the other only verbally indexed, is one 

more liberal, more consistent and more cer- which frequently arises in psychology and 

tain of their beliefs than the men. These dif- education. This study may serve as an il- 

ferences are presumably due to cultural lustration of the usefulness of analysis of 
rather than biological factors. The fact that variance in attacking such problems. 


the women of this sample have higher intel- In this study, the variable which is quan- 
ligence test scores indicates that a small part  tjtatively measured is the total liberalism 
{this apparent sex difference may be due to score on the Progressive Education Associa- 
the difference in intelligence. Several other tion test called “A Scale of Beliefs”. This test 
studies have shown that there is a positive has been described in detail in Section IV, to 
correlation between liberalism and intel- which reference may be made for informa- 
vence.. Murphy attempts to explain this tion about the particular test used. The 
rrelation by saying that more intelligent characteristics of the subjects for which less 
me oersons are bombarded by a different set of refined indications are available are: (1) 
ter- cultural influences than the less intelligent. It Best liked high school subject; (2) Mother’s 
is also possible that the more intelligent have education; (3) Father’s education; and (4) 
more test-taking skill. Father’s occupation. Of these, the first was 
{ 
a ; recorded by the subjects during freshman 
There is a much larger negative correla- week jn connection with the intelligence test- 
tion between uncertainty and liberalism than  jng program, and the other three were in- 
between uncertainty and conservatism. It cluded in their application blanks, which 
may be that those who hold liberal views de- they filled in before being admitted to the 
velop an emotional conviction to support their University. 
rh | opinions. The views of the conservative are 
usually traditional and do not need to be so 
r staunchly defended. Another possible ex- 
1:4} planation is that the test measures liberal- 
ism more accurately than it measures 
conservatism. 


The population included 437 white fresh- 
man students (288 women and 149 men) in 
the College of Education of the Ohio State 
University. These freshmen took the ‘Scale 
of Beliefs” test in connection with a survey 
course in education. 


enc 3. The correlations between liberalism in Ss an b that the est Whed hich 
con- race and other kinds of liberalism are lower + Cae BS eS See ee See ee oe 
ae than for any comparable set of correlations school subjects should be related to liberal- 
re This seems to indicate that college freshmen ism. For example, students who liked social 
th have “blind spots” in the field of race more studies best would be expected to be, on the 


whole, more liberal than students who pre- 
ferred mathematics. The approach here, 
however, is to examine the question by a 
statistical rather than a logical analysis. To 
ye ; ; ; deal with this issue, we first group the stu- 
4. There is a rather high correlation be- dents according to best liked high school 

tween liberalism in a certain field of thought subject, and compute a mean liberalism score 

ela- and consistency in that same field. for each group. These means will in general 
5. Consistency of response bears a positive differ from the mean for the entire sample, 
relationship to liberalism. It is also positively even if there is no relationship between the 
related to intelligence, but less closely than to basis for classification and the dependent 


frequently than in other fields of thought. 
The correlations in the field of democracy 
are highest, possibly because ‘“democracy”’ is 
here used as a rather inclusive concept. 





ee liberalism. variable; but if there is a relationship, the 
yeTal- . . . 
ae | & L. Murphy, & T. Newcomb, Experimental Socia) Variation in means among the groups should 


£¥, p. 930, New York, Harper, 1937, xi + 1121. be larger than would be expected from 





4 
: 
4 ; 
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chance. By the use of the F-test’ it is 
possible to determine the likelihood that a 
given set of differences between the group 
means and the mean of the total group would 
have arisen by chance. Evidence for a rela- 
tionship consists of an F-ratio for the data 
so large that its occurrence due to chance 
alone would be very improbable. Stated in 
other terms, the null hypothesis, which in 
this case would be that no relationship exists 
between best liked high school subject and 
liberalism, is considered to be disproved if 
the value of F turns out to be larger than 
would occur by chance in one per cent of 
experiments. 


The value of F is determined from the data 
by dividing the “mean square between 
means” by the ‘“‘mean square within groups.” 
In this ratio, the numerator measures the 
variance attributable to differences between 
the various groups and the denominator meas- 
ures the variance which is to be attributed to 
other factors than the one used in making 
the classification. 


In this study, the following grouping of 
best liked high school subject was used: so- 
cial studies, mathematics, biology, physical 
science, foreign language, English, vocational 
arts, fine arts, physical education, and com- 
mercial studies. Table I presents the results 
of the analysis of variance for this classifica- 
tion. For both sexes, the mean square be- 
tween groups is smaller than the mean square 
within groups, which tells us immediately that 


1 Snedecor, G. W. Statistical Methods. Collegiate Press, Inc., 
Ames, Iowa. 1937. 


TABLE I 


ANALYSIS OF VARIANCE: LIBERALISM VS. BEST 
LIKED HIGH SCHOOL SUBJECT 
Women* 


Sum of Degrees of Mean 
Squares Freedom Square 





Between groups... —_ 3.83 9 43 
Within groups____ 336.59 277 1.22 
Total _______ 340.42 286 
Men 


Sum of Degrees of Mean 
Squares Freedom Square 





Between groups... 5.33 9 .59 
Within groups____ 157.21 139 1.13 
Total _______ 162.54 148 


. Omitting the one no-data case on best liked 
subject. 


such a situation could easily arise by chance 
so that no F was computed. An examinatioy 
of the means of the groups does not lead ; 
any hypothesis for combining groups, by: 
rather supports the position that liberalisy 
and best liked high school subject as mess. 
ured in this study are not related. 

Tables II, III, and IV present the mean: 
and analysis of variance for mother’s edycy. 
tion in relation to liberalism. It wil! be ob. 
served that although the value of F js no 
significant when mother’s education is divides 
into six categories for either sex of students 
a highly significant value arises in the cas 
of the men when the grouping is into on) 


TABLE II 


MEANS* AND NUMBERS OF CASES: LIBERALIS! 
SCORE ACCORDING TO MOTHER’S EDUCATION 





Mother Students 
Women Men 
N M N M 
1. College graduate ______ 15 5.3 7 6.0 
2. Attended college but did 
not graduate _______ 59 5.6 20 528 
3. High school graduate__ 98 5.4 41 52 
4. Attended high school but 
did not graduate ____ 46 5.4 24 53 
5. 8th grade or less ______ 57 5.8 45 52 
Sf | eee 13 5.9 12 
ner 288 5.6 149 


*In this study, scores were coded according 
to the formula X = 10X’ + 4.5. The mea 
and variances reported are based on the tra: 
muted (X’) scores. 


TABLE III 


ANALYSIS OF VARIANCE: LIBERALISM ScoRi 
ACCORDING TO MOTHER’S EDUCATION 


Women 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups’ 8.03 5 1.61 


Within groups_ 332.72 282 1.18 





Total ____-- 340.75 287 
Men 
Degrees 
Sum of of Mean _ 
Squares Freedom Square f 
Between groups 11.43 5 2.29 


2.16" 


Within groups_ 151.11 143 1.06 


Total __.__-_ 162.54 148 
*5% point = 2.27. 
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TABLE IV 


ANALYSIS OF VARIANCE: LIBERALISM SCORES 
~~ AccoRDING TO MOTHER’S ATTENDANCE OR 
Non-ATTENDANCE AT COLLEGE 


Women 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups 48 1 48 
Within groups— 329.51 273 1.21 


329.99 274 


Men 
Degrees 
Sum of of Mean 
Squares Freedom Square F 
9.75 


Total 


3etween groups 
Within groups— 143.43 


Total ... 153.18 
1% point = 6.8. 


two categories. It should be noted that the 
second hypothesis tested was derived from 
the data of the sample, and is therefore open 
to question in spite of its statistical signifi- 
cance, unless it proves significant in a new 
sample.” 

The ease with which the initial classifying 
system may be telescoped in the use of 
analysis of variance represents an important 
advantage of this method for the empirical 
study of methods of classification. In this 
example it was found that for the particular 
sample, the essence of the grouping in terms 
of mother’s education was in the distinction 
between those who attended and those who 
did not attend college. By the use of quanti- 
tative criterion variables, it should be possible 
to determine the most economical system of 
classification for a variety of qualitative 
variables. 

The outcome of the combination of classifi- 
cations suggests a point which should be 
‘tated explicitly—the significance of an 
analysis of variance of the kind used here is 
not independent of the scheme of classifica- 
tion used for observations of the qualitative 
variable. If the @ priori hypothesis in the 
analysis of mother’s education in relation to 
liberalism had been the two-fold classifica- 
ton instead of the six-fold one, a significant 


relationship would have been found instead 


of an indeterminate one. 
hy Peirce, Charles §., Collected Papers, vol. I, pages 39- 


; Jf @ presentation of the dangers involved in forming 
ypotheses from the sample. 
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The analysis of variance of father’s edu- 
cation, when classified as in Table II, yielded 
mean square values which gave no evidence 
of a relationship between this variable and 
liberalism score. 


The analysis of variance of father’s occu- 
pation in relation to liberalism is presented 
in Table V. It is clear that among the men 
no well defined relationship exists. If, how- 
ever, for the women, the original code (which 
is modified from one used by the United 
States Employment Service) is telescoped into 


TABLE V 


ANALYSIS OF VARIANCE: LIBERALISM SCORE 
ACCORDING TO FATHER’S OCCUPATION 


Women 
Degrees 
Sum of of Mean 
Squares Freedom Square 
Between groups 11.24 8 


Within groups_ 329.51 279 
340.75 287 
Men 


Degrees 
Sum of of Mean 
Squares Freedom Square F 
Between groups 5.25 8 66 
Within groups— 157.29 140 1.12 


Total _... 


Total 162.54 148 


a threefold distribution as follows: (1) 
fessional, sub-professional, 
ministrative, supervisory, and _ unassigned; 
(2) Salespersons, clerical workers, service 
workers, and skilled craftsmen; and (3) Pro- 
duction workers and physical laborers, the re- 
sultant value of F is 4.42. For two degrees 
of freedom for the higher mean square and 
272 degrees of freedom for the smaller mean 
square, this is significant at the 5% level but 
not at the 1% level. 


Pro- 
technical, ad- 


SUMMARY 


This study presents several examples of 
the use of analysis of variance in dealing with 
the relationship between two variables, only 
one of which is quantitatively measured. 
With the systems of classification employed, 
none of the relationships turned out to be 
significant. Relationships between mother’s 
education and liberalism in men, and between 
father’s occupation and liberalism in women 
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are suggested. Evidence that the analysis of 
variance is not independent of the system of 
classification used is presented. 


VI. THE STUDY OF AN INTEREST 
QUESTIONNAIRE 
Reicn H. BItTNER AND EDWARD BoRDIN 


The Progressive Education Association re- 
cently devised an interest questionnaire the 
function of which was to serve as one of sev- 
eral instruments for evaluating the programs 
of schools participating in experiments in 
progressive educational methods. The ques- 
tionnaire was designed to give some measure 
of the extent to which the curricula of the 
schools were meeting the interests of the stu- 
dents. Interests may be variously classified. 
In this instance, the attempt was made to 
build an instrument which would make ap- 
parent an individual’s pattern of interests in 
various areas. 

The questionnaire was developed through 
the cooperation of a group of secondary 
school teachers representing various subjects. 
This group gathered together a large number 
of statements of activities in which they had 
observed their students engaged, and which 
they judged to give evidence of the students’ 
interest in particular school subjects or other 
cultural areas. For instance, ‘“‘To ride horse- 
back”’ was considered an evidence of interest 
in physical education or sports; “To take 
music lessons” was an indication of interest 
in music, etc. Many items were considered 
to evidence an interest in two or more areas. 
For example, “To discuss causes of war’’ in- 
dicated ‘Talking’, “Sociable”, ‘Mental’, 
and “Social Studies” interests. The present 
form of the questionnaire embodies 300 
items. ; 

The questionnaire was designed to be scored 
to distinguish nineteen areas of interest. Items 
are grouped according to the areas in which 
the teachers judged them to be expressions of 
interest. As already noted, many items are 
considered to give an expression of interest 
in more than one area. The individual who 
marks the questionnaire responds to each 
item with “Like” or “Dislike”, or makes no 
response, which is interpreted in the scoring 
as “Indifferent”. Thus three scores—the 
number of ‘Likes’, “Dislikes” and ‘“‘Indiffer- 
ents’’—may be found for each of the nineteen 
areas. The nineteen areas have been further 








grouped on a profile graph into 
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larger areas of interest named ‘Curricula; 


“Verbal”, “Aesthetic”, and “Sociable” for +h, 


facilitation of guidance, although no compox. 
ite scores are reported for these four area 

The present study is concerned with day, 
gathered during the use of this questionnais: 
by the College of Education of The (bh; 
State University in the fall quarter of 192: 


for the purpose of evaluating their interes. 
as a basis for educational and vocation 
guidance. (It is to be noted that this pr 
posed use of the questionnaire involves a dij. 
ferent approach from that of the original jp. 
tention of the Progressive Education Associa. 


tion.) A number of questions arose 
consequence of the administration o/ 


questionnaire. This study deals with three 


these questions. 


QUESTION I 


Are there any primary interest factors u- 
derlying the nineteen areas of interest mea: 


ured by the questionnaire? 


The answer to this question has pract 


as well as theoretical significance. If it 


fundamental interest factors, then it sh 


be possible to score the questionnair 
these principal factors and to deriv: 
scores for any of the nineteen separat 


f 


terests by a linear combination of the s 
for the principal factors. This would great) 
simplify the scoring of the questionnaire. I 


answer will also have a bearing on thi 


wer 
“W 


found that there was a small number 


A factor analysis of the correlation matrices 


for “Likes” and “Dislikes” was made. 
“Simplified Multiple Factor Method 


The 


(e- 


veloped by Thurstone’ was used. For reasons 
which we will develop, the “Like” and “D's 
like’ intercorrelations were analyzed sep. 


rately. 
Two factors are distinguished by 


analysis. Since the two factors reduced 


residual correlations to approximately wh! 


1 
the 


the 
h 


would have been expected by chance, 

° ° lat 
seems that it is not necessary to postu 
more than two factors to account for the tade 


of intercorrelaions. The factor loadings & 
presented in Table II indicate that the '™ 


factor is overwhelmingly important 


rs 


The 


factor loadings for the first factor are all pos 


Math 


‘ Thurstone, L. L. A_ Simplified Multiple Factor 
University of Chicago Bookstore, 1933, 25 pp 
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tive and relatively high for both “Likes” and 
“Pislikes”. The second factor loadings for 
both “Likes” and “Dislikes” with but few 
exceptions are relatively low and account for 
very little of the variance. 


At this point we may speculate as to the 
probable nature of the two factors. The first 
factor loadings for “Likes” which seem most 
characteristic of the factor are derived in the 
order of their significance, from the interests 
labelled “Talking”, ‘Reading’, ‘““Humanitar- 
ian”, “Social Studies”, “‘English’’, ‘Sociable’, 
“Mental”, “Writing”, and “Leadership”. It 
is difficult to assign a name to this factor but 
it seems probable that an interest in working 
with ideas may characterize this factor. 
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ture of the factors are speculative and merely 
tentative. These conclusions would require 
further study of an empirical nature for 
verification. 

The fact that many items are common to 
two or more interests raises another question. 
Are the factors found by means of the factor 
analysis only pseudo-factors resulting from 
the effects on the intercorrelations caused by 
certain items being common to the interests 
correlated? In order to get at this problem 
it was necessary to determine how much of 
the correlation was due to the presence of 
common items. 

A formula was developed for estimating 
the amount of the spurious correlation. The 
correlation between two tests is given by 





(A+ B + X) 
Tiaabs + Xx) — 
VTarsTaOn . TanOaOn 4 el x(X - 1)FxOX - 44 %xxOx0x (1) 
= = as a = — $n = = I 
2 2 2] 
Woe. “ & Tons + ox — 2 (Tar FaF a ‘es ae Vx¢x - 1) TxOx - 1) 
\ oA" — on” + 





The first factor loadings for “Dislikes” 
give essentially the same picture as we found 
for “Likes”. “Theatre Arts” is added to the 
interests which appeared to constitute the 
first factor for “Likes”. We may again as- 
sume that an interest in working with ideas 
may characterize this factor. 

[he second factor loadings for both 
“Likes” and “Dislikes” indicate that it is not 
of great significance. However, we may con- 
jecture as to what this factor may be. The 
second factor loadings for “Likes” which are 
most characteristic of the factor are called 
Manipulative”, “Industrial Arts’, and “Fine 
\rts’. Table III shows that “Manipulative” 
is closely allied with “Industrial Arts” and 
Fine Arts’ as a result of the method of 
scoring items. We may postulate that the 
common factor here is probably an interest 
in working with things. 

The second factor loadings for “Dislikes” 
which are most definitive of the factor are 
called “Industrial Arts”, “Manipulative”, and 
science, Engineering and Mathematics”. 
lable IIT again shows that “Manipulative” 
s closely related to the other two by the 
method of scoring items. It seems that we 
may again assume that the second factor is 
rates by an interest in working with 

ings. 

It should be emphasized at this point that 
these considerations with regard to the na- 





~ + ox? + 2(ranonon +... + 9x yx - oxox - 1) 


where a, b, . . . x are items of one test and 
A, B,. . . X are items of the other test. 

Now assuming that the o’s are all equal 
and the items are independent, equation (1) 
may be written 

pa eT + + Mex. (2) 

VPV_I 
where p is the number of items in the first 
test and g is the number of items in the 
second test. 

All numerator r’s are either 1.00 or 0.00. 
If the item is common to both tests, then 
r== 1.00 and if the item occurs in one test 
and not the other, then r = 0.00 in accord- 
ance with our assumption of independence of 
items. Then equation (2) may be written 

n 


VP Vd 
in which m is the number of items common 
to the two tests. 

Table III shows the estimated spurious 
correlations determined when equation (3) is 
used. When these values are compared with 
the correlations shown in Table I, it is seen 
that there is some correspondence between 
the magnitudes of these correlations and the 
estimated amounts of spurious correlation. 
Now if there were a linear relationship be- 
tween these two series of values, we could 


(3) 
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conclude that the individuality of the factors 
~ largely determined by the items which are 
-ommon to the interests correlated. Figure 1 
‘hows that some degree of linearity is evident 
«chen the intercorrelations of the interests 
most characteristic of the first factor for 
Likes” are plotted against their correspond- 

. estimates of spurious correlation due to 
‘mon items. However, it is also quite ap- 
sarent that the individuality of the factor 
‘ not solely determined by the spurious cor- 
-elation, but that the factor has some sig- 
ificance in its own right. We can only 
-onelude that the factor is partially the re- 
it of the common items, and it is left for 
further study to determine the true nature 
ind magnitude of the factor. 
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QUESTION II 


Are the “Like” and “Dislike” scores posi- 
live and negative expressions of the same 
trait? 

The answer to this question will determine 
whether it is necessary to score the question- 
naire on both “Likes” and “Dislikes” or 
whether scoring on one will be sufficient. It 
was also intended that if we found only one 
score to be required some recommendation 
should be made as to which score was the 
more meaningful. It was these considerations 
that led us to analyze separately the “Like” 
and “Dislike” intercorrelations. 
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Table Il shows that the factor composi- 
tions for “Likes” and “Dislikes” are similar 
to a marked degree. The first factor loadings 
for “Dislikes” are in general higher than 
those for “Likes” but there is apparently 
rather close correspondence between the two 
with respect to rank order. Rank order co- 
efficients of correlation were computed and it 
was found that p = .896 + .032 for the first 
factor loadings and .577 + .108 for the sec- 
ond factor loadings. Since “Like”? and “Dis- 
like” scores are mutually exclusive responses 
to the same sets of items, it seems reasonable 
to assume that these results indicate that 
such scores are positive and negative expres- 
sions of the same trait. A really definite an- 
swer to this question can be obtained only by 
treating “Like” and “Dislike” scores in the 
same correlation matrix. 


For further evidence bearing on our ques- 
tion we turn to the correlations between 
“Likes” and ‘Dislikes’ as shown in Table II. 
These correlations (on the diagonal) are all 


negative, and rp = — .508. Now if these 
correlations were of the same order as the 
reliabilities of the variables we would have 
tangible evidence that “Likes” and “Dislikes” 
are positive and negative measures of the 
same trait. Estimated reliabilities as shown 
in Table II were computed using Kuder and 
Richardson’s ‘‘Case IV Method.’* The fact 
that the reliabilities greatly exceed the cor- 
relations between “Likes” and “Dislikes” 
might seem to indicate that scores for 
“Likes” and “Dislikes” are not positive and 
negative expressions of the same trait. How- 
ever, let us consider the correlations between 
“Indifferents” and “Likes” and ‘‘Dislikes”’. 
Table II shows that the correlations between 
“Likes” and “Indifferents” are consistently 


higher than _ those between “Dislikes” 
and “Tndifferents”’. In fact, rip1 = — .449, 
Tyr == — .121, and rp == — .508. This in- 


dicates that “Like” and “Dislike” are at op- 
posite poles of a continuum but that the cor- 
relation between “Likes” and “Dislikes” is 
limited by the fact that “Indifference” also 
lies opposite to “Like”. Therefore we can 


not expect rip to be of the same order as the 
reliabilities, though in view of the data 
already cited, it is highly probable that 7,» 
corresponds closely enough to the reliabilities 

2 Kuder, G. F. and Richardson, M. W. “The Theory of the 


Estimation of Test Reliability.” 
151-161. 


Psychometrika, 1937, 2, 
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to add some further evidence that scores for 
“Likes” and “Dislikes” represent positive and 
negative expressions of the same trait. 

We may then conclude that the second 
question has been answered affirmatively. It 
also seems apparent from the factor loadings 
in Table II that “Dislikes” gives a better 
measure than “Likes”. This is especially true 
for the first factor. 


Question III 


What is the significance of the “Indiffer- 
ence” score? 

There is some indication that the “Indif- 
ference” score yields a negative expression of 
the trait which is expressed positively by the 
“Like” score. By following the same reason- 
ing that we pursued in the analysis of the 
“Like’—‘Dislike” relation we find that the 
combination of an 7,1; —.449, which is 
limited by the rp == — .508, and high re- 
liabilities for both “Likes” and “Indifferents” 
seems to give statistical evidence that “Likes” 
and “Indifferents” are to a certain extent 
positive and negative expressions of the same 
trait. 

The relation existing between “Dislike” 
and “Indifference” scores is manifestly differ- 
ent from that between “Like” and “Indif- 


ference” scores. As indicated by ror = — .121 
there is very little relationship between these 


two series of scores. When rp; is compared 
with the estimated reliabilities as shown in 
Table II, there seems to be evidence to indi- 
cate that “Indifference” and “Dislike” 
scores are not measures of the same trait. 
This lack of relationship between “Indif- 
ference” and “Dislike” scores seems to point 
to the possibility that the “Indifference” 
scores may be a measure of a different 
psychological trait from that measured by the 
“Like” and “Dislike” scores. As we have 
shown, “Like” and “Dislike” scores are ap- 
parently positive and negative expressions of 
the same trait. Now it was also shown that 
statistical evidence points to the conclusion 
that “Like” and “Indifference” scores are 
positive and negative expressions of the same 
trait. However, if this were true we should 
expect that “Dislike” and ‘Indifference’ 
scores would be closely related. Since that is 
not the case, it is highly probable that the 
relationship found between “Like” and “In- 
difference” scores is a function of the closed 
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system of scoring and not a true psychologic, 
relationship. It then appears to follow thy 
the “Indifference” scores may possibly repre. 
sent a different psychological trait from thy 
measured by the “Like” and “Dislik. 
scores. This supposition would necessari)y 
have to be verified by further study. A facto; 
analysis of the intercorrelations of the “|p. 
difference” scores on the nineteen posited jp. 
terests would give further evidence on thi 
point. 

It is to be noted that Table IT shows th: 
estimated reliabilities for the “Indifference 
scores to be excessively high in compariso; 
with those of the “Like” and “Dislike 
scores. These excessively high estimated r. 
liabilities are probably due to the fact thy 
the standard deviations for “Indifference” 4 
shown in Table II are in every case higher 
than those for “Likes” and “Dislikes”, This 
in turn results from the method of scoring 
which requires the sum of the “Likes”, “Dis. 
likes”, and “Indifferents” scores to equal ; 
constant, i. e. the total number of items. Ip 
such a closed system of scoring, the standard 
deviation of the “Indifference” scores equals 
the standard deviation of the composite 
scores obtained by adding the corresponding 
“Like” and “Dislike” scores. 


SUMMARY AND CONCLUSIONS 


The interest questionnaire recently de- 
veloped by the Progressive Education Asso- 
ciation was given to 462 freshmen in the (ol 
lege of Education of The Ohio State Univer- 
sity during the fall quarter of 1938. Three 
scores, namely the number of “Likes”, “Dis 
likes” and “Indifferents”, were obtained for 
each person on nineteen interests which the 
questionnaire was designed to distinguish 
The administration of the questionnaire gave 
impetus to the present study, in which ve 
have attacked the following questions: (1) 
Are there any primary interest factors under- 
lying the nineteen areas of interest measured 
by the questionnaire? (2) Are the “Like 
and “Dislike” scores positive and negative 
expressions of the same trait? (3) What & 
the significance of the “Indifference” score’ 

The statistical evidence shows that there 
are two primary interest factors which the 
questionnaire measures. The factor loading 
for both “Likes” and “Dislikes” show that 
the first factor is of much greater importanct 
than the second factor. An analysis of the 
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trinsic relation between interests resulting 


" j-om the fact that many items are scored on 


‘wo or more interests shows that the first 
‘actor is partially determined by the spurious 
correlation resulting from this method of 
.oring. However, the factor seems to have 
come degree of individuality in its own right. 
Speculation as to the nature of the factors 
-ads to the tentative conclusion that the first 
‘actor represents an interest in working with 


q eas and the second factor an interest in 
working with things. 


Statistical evidence points to the conclusion 
that “Likes” and “Dislikes” are positive and 
negative expressions of the same trait. The 
lose correspondence between the factor load- 
nes for “Likes” and “Dislikes”, and a com- 


 oarison of the negative correlations between 


Likes” and “Dislikes” with their estimated 


' reliabilities, lead us to this conclusion. There 


» seems to be adequate evidence to show 


that with the present method of scoring, the 


Dislikes” score gives a better measure than 


the “Likes” score. 


The study seems to indicate that the “In- 
ifference’”’ score may represent a measure of 
| different psychological trait from that 
easured by the “Like” and “Dislike” 
Our data indicate that “Like” and 
Indifference” scores may be positive and 


scares 


| negative measures of the same trait. It also 


' shows that “Dislike” and “Indifference” are 


not measures of the same trait. The negligible 
relationship between “Dislike” and “Indif- 
ierence’, instead of the close relationship that 
would be expected on the basis of the 
previous finding that “Likes” and “Dislikes” 
are inverse measures of the same trait, indi- 
cates that “Indifference” may be a measure 

i a different psychological trait from that 
measured by “Likes” and “Dislikes”. The 
adication that “Likes” and “Indifierents” 
are positive and negative measures of the 
same trait is apparently an outcome of the 
closed system of scoring. 

It is clearly evident that the questionnaire 
should not be considered to give measures of 
nineteen unitary and independent interests. 
since the factor analysis shows that only two 
unitary interests are measured by the ques- 
tionnaire, it is clear that many of these nine- 
‘een interests are closely related and are in 
4 certain sense measures of the same thing. 
This fact should be taken into consideration 
using the questionnaire for guidance, and 
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it is suggested that a regrouping of the nine- 
teen interests on the profile graph to conform 
with the factor analysis might be in order. 


The results of the factor analysis also in- 
dicate that the scoring of the questionnaire 
could be simplified. It should be possible to 
score for the principal factors instead of the 
separate interests, and then derive the score 
for any given interest by a linear combination 
of the factor scores, using the proper weight- 
ings for the given interest. However, the 
practice of giving the items equal weights for 
several interests has been shown to influence 
not only the validity of the factors but also 
their composition. It is probable that with 
differential scoring weights for the items, a 
different and perhaps more valid factor pat- 
tern would result. It seems to be indicated 
that the question of differential weighting of 
items should be investigated before an at- 


tempt is made to develop a simplified method 
of scoring. 


The evidence that the questionnaire now 
measures preponderantly one unitary factor 
and to a lesser degree a second seems to in- 
dicate that careful consideration should be 
given to the selection and validation of items. 
It may be that in the population studied only 
two unitary interests are operating, but it 
seems more likely that there are others which 
the questionnaire does not measure. The 
uniqueness of some of the interests would 
also point in this direction. These other fac- 
tors, if found, would greatly extend the use- 
fulness of the instrument in guidance. 


It is apparent that “Like” and “Dislike” 
scores are not both necessary to the measure- 
ment of the interests with the present scoring 
system, but that one or the other will suffice. 
A consideration of the factor loadings for 
“Likes” and “Dislikes” indicates that the 
“Dislikes” scores should be used if a choice 
is to be made between them. 


Finally, it is strongly urged that a more 
intensive study be made of the “Indifference” 
scores to determine the nature of the 
psychological trait which they seem to meas- 
ure. In view of the fact that the closed 
system of scoring leads to complications in 
interpreting the “Indifference” variable, it 
seems advisable to investigate the possibilities 
of a different system of scoring which would 
eliminate these difficulties. 
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VII. MEASURING FRESHMAN 
COLLEGE STUDENTS’ ABILITY TO 
THINK IN TERMS OF SELECTED 
SOCIAL PROBLEMS 


WILLIAM J. JONES 


Included in the battery of diagnostic tests 
which all freshmen in the Ohio State Univer- 
sity College of Education took in conjunction 
with a first quarter Survey of Education 
course during 1938, was a test on social 
problems devised by the Evaluation Staff of 
the Progressive Education Association. It was 
the purpose of this test to secure evidence on 
various aspects of students’ thinking in situa- 
tions where social values and beliefs have a 
strong, if not determining, effect on the 
quality of thinking. 

The test was composed of six problematic 
situations dealing with such problems as the 
use of labor saving machines, housing, taxa- 
tion, racial discrimination, freedom of speech, 
and health security of workers in factories. 
Each problem was followed by three or four 
possible courses of action, one or more of 
which were to have been checked by the stu- 
dents as appropriate “solutions” of the prob- 
lem. Beneath the courses of action about 20 
or 25 typical reasons were supplied, some of 
which might be used to justify the course (or 
courses) of action checked by the students. 
For each of the six problems in the test, the 
students were to check appropriate courses 
of action and to support their conclusions 
with reasons by marking those which they 
felt to be correct. 


Typical of the kinds of problem situations 
included in the test is the one which follows: 


PROBLEM I 


Cotton has been picked by hand, which 
is a slow and expensive process. Recently, 
the Rust brothers invented a machine to do 
this work. It would pick in 714 hours as 
much cotton as one handpicker could pick 
over a whole season of eleven weeks. The 
cost of production of cotton could be reduced 
from $14.52 to $3.00 per bale. Today this 
machine has not been placed on the market. 


Directions: Check the course (or courses) of 
action which you think should be followed 
in this situation. 
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Courses of Action: 


——A. 


——B. The machine should be manufa; tured 
and sold under some form of goy. 
ernment control and provisions mag; 


—C. 


——D. 


The invention should be made ays'! 


Vdll- 
able for unrestricted manufacture 


sale of the machine. 


for establishing in other jobs the c: 
ton pickers who are thrown out 
work. 


Workers and cotton growers shou); 


form a cooperative and should 
the profits from it to take care 
the people thrown out of work | 
the machine. 

The machine should not be put : 
use at the present time at al! 


Directions: Choose the reasons which yoy 
would use to support your course | 
courses) of action and check them in th: 
space provided. 


Reasons: 





I. 


. When labor saving machines are 10 


In business the efficiency of produc- 
tion should be considered ahead of 
anything else. 


. Uncontrolled use of the machine 


would give an advantage to a few 
large landowners in the South over 
the poor landowners. 


. The labor problems of the South ar 


for the South to solve. 


. One should consider human eiiect 


ahead of business efficiency. 


. Uncontrolled introduction of a labor 


saving machine throws large num- 
bers out of work. 


. Men who are clever enough to in- 


vent useful machines should have 


the opportunity to make profits o 
those inventions. 

troduced, the people who are (is 
placed by them find work in making 
the new machines. 


. People who are directly involved 


cotton production can better pla 
for the use of the machine. 


. Machines have worked too muc 


harm on people already. 


. People thrown out of work by the 


introduction of the mechanical © 


use 


Compt 


’ 
» 
. 
i 
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ton picker will be on relief rolls and for each problem), and that they checked 42 
thus add to the financial burden of reasons to support their views, of which 31 
the government. were sound reasons. The comprehensiveness 

* * * ratio of sound reasons per conclusion was 
14. Society should not be deprived of 4-5: That the students’ thinking in the field 
anything that might improve its of social values is somewhat confused is evi- 


gov- work and the products it uses. denced by the fact that of the 42 reasons 
made : P which the typical student checked, six of 
€ Cot- Some of the statements above represent them on the average actually contradicted 


ints of view which expressed a concern for the courses of action which the student 
mocratic and human values, freedom of checked. The mistake is akin to supporting 


should -yeech and social control, while others de- black with white. The students also were 
d use fended special privilege and individualism as guilty of using irrelevant and invalid reasons 
are of values. The tests were not scored on a right to support their points of view. Table I fur- 


and wrong basis, but the reporting of student ther indicates that the students’ patterns of 
 »erformance was in terms of the comprehen- value are highly democratic, with a slight 

ut t siveness of the students’ thinking (their abil- tendency toward compromise, but by and 
ty to support courses of action with sound large, the students chose no undemocratic 

reasons), as well as the validity, relevance, courses of action, and only four out of a 


h you ind consistency of the reasons chosen in possible total of 36 undemocratic reasons, 
e (or terms of the courses of action which were ii. e., special privilege and individualism. 
in the q checked. In addition to this, the dominant In addition to presenting this picture of 


pattern of values expressed through the stu- the democratic pattern of values rather com- 
dents’ choice of conclusions and reasons was prehensively supported by sound reasons, 
lescribed. which characterized the social attitudes of 





ney? lable I contains the median scores and the the group of over 500 freshmen, this report 
ee range of the scores for the entire group of has the additional purpose of presenting the 
545 education freshmen who took the test. inter-correlations between the students’ 
achine This table indicates that, on the average, performance on the sixteen parts of the test, 
" lew the students chose six courses of action (one as well as the relationships between the sev- 
aan 
ith are TABLE I 
MEDIANS AND RANGEs OF SCORES ON A SOCIAL PROBLEMS TEST 
effect Highest 
Median Range Possible 
+ Score 
a labor Comprehensiveness 
- num. 1. Number of courses of action checked __- 6 3-14 22 
2. Number of reasons checked_____________- : 42 13-125 144 
3. Number of sound reasons checked_________- ; 31 6-85 144 
to In- 4. Average number of sound reasons per conclusion _. 4.5 0.5-12.7 Sat 
1 have Confusion of Thinking 
fits on 5. Number of reasons inconsistent with course of action 6 0-42 144 
6. Per cent of irrelevant reasons___._-______- ane 22 0-67 100(9) 
‘. Per cent of invalid reasons__._._._._________- , dc oa 0-80 100(10) 
ar dis DOMINANT VALUES & VIEWPOINTS 
ki Values Expressed Through Conclusions 
making 8. Number of democratic conclusions___________- ; 4 0-9 a 
9. Number of undemocratic conclusions. __________- 0 0-4 7 
Ived in 10. Number of compromise conclusions. ___________- 2 0-5 6 
t plan Values Expressed Through Reasons 
11. Defense of special privilege. ____.....-.------- 3 0-25 27 
12. Defense of human values. ____.._.....-.-.--.----- 14 2-28 28 
much 13. Defense of social control in behalf of human values and 
democratic principles............................ 3 0-7 7 
14. Defense of individualism___________________________. 1 0-8 9 
by the 15. Defense of democratic principles._.__.._______________ 10 1-24 24 
cal cot- 16. Defense of compromise for sake of expediency... _____- 1 0-9 9 
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eral parts of this test and such data as rank 


in high school class, intelligence, and 
academic achievement as measured by 
grades. 


An analysis of such a table of intercorrela- 
tions would provide one basis for answering 
such questions as the following: 

Who is the more comprehensive in his 
thinking, the person with the democratic or 
the undemocratic outlook? 

Is the tendency to make one type of error 
related to the tendency to make other types 
of errors? 

Is one who holds a democratic point of 
view more likely sometimes to take a com- 
promise position than one who holds an un- 
democratic point of view? 

Which outlook do the students who make 
the highest grades in school and the highest 
scores on an intelligence test take? 

Table Il presents the inter-correlations be- 
tween these variables. It indicates that those 
students who chose democratic courses of 
action tended less often to choose compromise 
positions than did those students who chose 
undemocratic courses of action. The table 
further shows that the tendency to make one 
type of error possible in the test is associated 
with the tendency to commit other types of 
errors possible in the test. The tendencies 
to use inconsistent and invalid reasons seem 
to be the more closely associated. To facil- 
itate interpretation this master table has been 
subdivided into significant parts. 

Table III presents the relationships which 
exist between the sixteen test variables and 
the following: (1) the student’s rank in his 
high school class as measured by a lowest 
third, middle third, and upper third classifica- 
tion; (2) the student’s intelligence as meas- 
ured by the Ohio State Psychological Exam- 
nation; (3) academic achievement as meas- 
ured by his first quarter grade-point average; 
and (4) grades in the Survey of Education 
course when recorded on a letter basis from 
A to E. 





\Vo + A 

The interesting fact about Table IIT js thy 
there are no high correlations, either negatiy. 
or positive, included in it. The r of .23 bp. 
tween intelligence and the comprehensivene: 
score is the highest coefficient of correlatio; 
found in Table III. 

Although the coefficients of correlation 
small, it is nevertheless significant 1) 
the pattern of positive and negative correla. 
tion which exists between the columns. With 
the exception of columns 1 and 2, the sin 
of the quantities within a given column 
all the same. It is significant to note that th 
students who chose democratic courses 9! 
action (column 8), and supported them cop. 
sistently with reasons which defended humar 
values (column 12), social control (colum 
13), and democratic principles (column 1: 
tended to be those students with higher jp. 
telligence test scores (row 2), higher stand- 
ing in their high school graduating classes 
(row 1), and who secured the better grades 
(rows 3 and 4). This conclusion is supported 
not so much by the fact that the correlatior 
coefficients are high, as by the fact that for 
each column mentioned above the r’s are in- 
variably positive across all of the four rows 
The fact that there is a consistent invers 
relationship between the tendency to check 
undemocratic (column 9g), and compromise 
(column 10) courses of action, utilizing rea- 
sons defending special privilege (column 1 
individualism (column 14), and compromise 


in one’s high school graduating class, having 
a high intelligence test score, and the ability 
to secure high grades, tends to emphasize the 
relationships noted above. 

Table IV shows the relationships between 
the various courses of action and the different 
types of reasons which were used to suppor 
the courses of action. The table indicates 
that there is a fairly high correlation (rang: 
ing from .38 to .43) between the tendency 
to choose democratic courses of action an 
to support them with reasons which affirm 


TABLE IV 
RELATIONSHIP BETWEEN VARIOUS COURSES OF ACTION AND CERTAIN KINDS OF REASONS 


Special Human 

Courses of Action Privilege Values 
Democratic .............. —.14 43 
Undemocratic _...________ .27 —.18 
Compromise ~_._.._______ 36 —.08 





Reasons 
Social Individ- Democratic Com 
Control ualism Principles promise 
40 .05 38 —.02 
—.12 .27 .02 5 
—.20 12 —.11 25 
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emocratic principles and defend human 
4jues and social control. More significant 
perhaps are the positive correlations between 
‘he tendency to choose undemocratic courses 
{ action and the tendency to choose reasons 
which defend individualism and _ special 
orivilege as social values. Interesting, too, 
s the tendency for those who chose reasons 
arming special privilege to choose com- 
oromise courses of action. 

lable V indicates the relationships between 
the tendency to make certain types of errors 
and the tendency to choose certain courses 
of action and various types of reasons. The 
table shows that there is almost no relation- 
ship between the kinds of courses of action 
one chooses and the kinds of errors in think- 
ing which are made. The table further in- 
dicates that those students who chose reasons 
defending special privilege, individualism, 
and compromise positions tended to be 
more inconsistent in their thinking than 
those who chose reasons which supported 
democratic principles, human values, and so- 
cial control. Table V likewise shows that 
there is a fairly high correlation (.52) be- 
tween the use of “human values” reasons and 
the tendency to employ reasons which are ir- 
relevant to the courses of action selected. 
Row 3 in Table V shows that those students 
who chose reasons defending special privilege, 
individualism, and democratic principles 
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tended more often to use invalid reasons than 
those students who chose other types of 
reasons. 

Table VI shows the inter-relationships be- 
tween the use of the various types of social 
reasons included in the test. It indicates that 
the tendency to employ reasons which sup- 
port special privilege is associated with the 
tendency to employ reasons which support in- 
dividualism as a basis for social action. Table 
VI also shows that those students who chose 
reasons which defended democratic principles 
likewise tended to choose reasons which 
affirmed social control as valid social values. 

This study has indicated that the College 
of Education Freshmen at the Ohio State 
University tend to be democratic in their so- 
lutions of social problems, and that they are 
able, on the average, to supply four or five 
sound reasons for each of their courses of 
action. The pattern of values exhibited in 
the types of reasons which they employ ex- 
pressed a concern for the defense of funda- 
mental democratic principles, human values, 
and social control. The _ inter-correlations 
show that the “better” students (high grades, 
high intelligence test scores, and high rank in 
high school graduating class) tended to 
choose democratic courses of action and 
democratic reasons more often than they 
chose undemocratic or compromise courses of 
action and reasons. 


TABLE V 


RELATIONSHIP BETWEEN VARIOUS KINDS OF ERRORS AND DIFFERENT TYPES OF COURSES 
OF ACTION AND SOCIAL REASONS 


Reasons 
Errors Courses of Action Demo- 
Demo- Undemo- Com- Special Human_ Social Individ- cratic Com- 
cratic cratic promise Privilege Values Control ualism Principles promise 
Inconsistent .03 13 13 50 .29 22 44 36 46 
lrrelevant__ .17 06 —.06 .05 52 17 ae .30 .20 
Invalid 10 17 02 BD .26 on 31 .30 Ps - 
TABLE VI 
INTER-RELATIONSHIPS BETWEEN THE USE OF VARIOUS TYPES OF SOCIAL REASONS 
Reasons 
Reasons Special Human Social Individ- Democratic Com- 
. ae Privilege Values Control ualism Principles promise 
Special Privilege _______ 
Human Values _________ 01 
Social Control] __________ 16 Al 
Individualism __________ 49 .06 10 
Democratic Principles _-_ 30 40 58 .28 
Compromise ____________ A4 .20 10 .29 12 
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VIII. RELATIONSHIPS OF TEST 
SCORES OF EDUCATION 
COLLEGE FRESHMEN 
TO GRADES IN 
SELECTED 
COURSES 


ARTHUR C. CAHOW 


The purpose of this study is to determine 
the relationships which exist between the 
marks in various courses and the scores ob- 
tained from the various tests. 

Course marks were available for English 
401, Composition; Botany 4o1, First Course 
in Botany; Psychology 401, Elementary Psy- 
chology; History 403, American History to 
1852; Geography 401, Elementary Geog- 
raphy; Music 423, First Course in Pub- 
lic School Music; Fine Arts, Freehand Draw- 
ing, and Zoology 401, Elementary Zoology. 
Inasmuch as this investigation presupposes 
some degree of choice of the courses selected, 
English 401 was eliminated because it is re- 
quired of all students; Zoology 401, Psychol- 
ogy 401, Geography 401, Music 423, and 
Zoology 401 because the number of cases was 
small, being considerably less than fifty in 
each instance. 

The courses selected for this study were: 
A) Botany 4o1, First Course in Botany; B) 
History 403, American History to 1852; and 
C) Fine Arts 421, Freehand Drawing. His- 
tory 403 might be defined as a partial elec- 
tive, since it is required of students in the 
Elementary Teachers’ curriculum of the Col- 
lege of Education. These three courses are 
assumedly quite different. One is a natural 
science, one a social science, and the other a 
skills course in an esthetic field. With such a 
selection of courses, the liability and asset 
values of the variety of backgrounds reflected 
by the tests may be investigated. 

The test scores used more selected from 
those described in the preceding studies. Not 
all of the subscores were used. Selection was 
made on the basis of the statistical evidence 
already presented and on the judgment of 
those using the test results. 

Scores of the O. S. U. Psychological Test 
are expressed in percentile ranks, scores of 
other tests in percentage, and course marks 
as letters A, B, C, D, and E. Hollerith 
technique was employed for tabulation. 

Correlation coefficients were computed for 
each of the test scores and grades for each 
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of the selected courses. The means apg 
standard deviations were obtained for the te; 
scores of those who did and of those who dj 
not elect each course. 

The essential results of this study ar 
shown in two tables. Table I shows correla. 
tions of each of the three selected course 
with each of the sub-tests used in this stu) 
Table II shows the means and standard deyi.. 
tions of the scores on each test for those wh 
elected each of the three courses and for thos 
who did not elect these courses. 

In general, correlation coefficients are 
low numerical value. However, assuming thy 
validity of these tests and the represents. 
tiveness of the samples, those to be discussed 
here are significant with respect to a group 

One might arbitrarily select as a significar: 
value of a correlation coefficient only on: 
which has less than one chance in one hur- 
dred of being drawn from a universe where 
the true value of the correlation coefficient is 
zero. Any correlation coefficient which is a 
least 2.236 times the standard error of a cor- 
relation coefficient of zero, with the same 
number of observations, may then be consid- 
ered significant. The minimum significant 
value for the Botany correlations is 0.17 
for the History correlations, 0.284, and for 
the Fine Arts correlations, 0.293. 

For convenience in presenting the results 
the conclusions will be discussed in terms 
the variables composing the study. 


O. S. U. PsycHoOLociIcAL EXAMINATION 


The coefficient of correlation of the Psy- 
chological Examination with Botany grades 
is 0.526 and with History grades, 0.5% 
These coefficients are among the _ highest 
found in this study. One might be inclined 
to believe, a priori, that the numerical rela- 
tionship should be reversed, inasmuch as the 
Botany grades are computed from the scores 
of five standardized tests, whereas the His 
tory grades are computed from the results 0 
subjective type tests and grades on term 
papers. The correlation of the Psychologica 
Test and Fine Arts grades is 0.270. 4 
popular belief is that students who ele 
“manual” or “skills” courses are of lower it 
telligence than are those who major © 
academic work. Intelligence is here defined 
that trait which is measured by the 0. S. | 
Psychological Test. Examination of the mean 
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TABLE I 


COLLEGE FRESHMAN TESTING PROGRAM 


CORRELATION COEFFICIENTS OF TEST SCORES AND COURSE GRADES 


0. 8. U. Psychological Examination 


Social Problems Test 
No. of Sound Reasons . 
Average No. of Sound Reasons 
No. of Inconsistent Reasons_ 
No. of Democratic Conclusions 
No. of Compromise Conclusions 
Defense of Special Privilege 
Defense of Individualism 
Defense of Democratic Principles 
Defense of Compromise 


Social Attitudes Test 
Liberalism 
Democracy 
Nationalism 
Militarism 


Total 


Conservatism 
Democracy 
Nationalism 
Militarism 
Total 


Uncertainty 
Democracy 
Nationalism 
Militarism 
Total 


Consistency 
Democracy 
Nationalism 
Militarism 
Total 


Interpretation of Data Test 
General Accuracy 
Relative Values (Total Score 
Insufficient Evidence 
Obviously True or False 
Caution—Understatement 
Beyond the Facts 


Contemporary Affairs Test 


International Events 
Total Score—Part I 


Part II, Contemporary Culture 
Books. 


Music and Radio. ___ 
Movies 





Botany 401 


. 526 


9-9 
mir 
. 230 
. 120 


.110 


026 
090 
. 061 
176 
016 


247 
.318 
230 

293 


.132 
. 268 
73 


.201 


. 140 
.O88 
. 128 
144 


. 367 
. 304 
. 230 
. 300 


407 


. 170 


. 438 


. 264 


. 154 


. 108 


. 190 
. 230 


. 106 
. 073 


. 060 


History 403 
. 580 


223 
424 
362 
. 050 
062 
157 
. 224 
024 
066 


. 584 


599 


. 522 
. 371 

. 093 
. 442 


. 306 


. 227 
. 330 


. 170 
—. 039 
. 130 
. 202 


Fine Arts 421 


270 


. 185 
.148 
142 
209 
. 023 
136 
198 
.072 


104 


. 286 
. 352 
~ 050 
. 199 


177 
. 265 
058 
119 


098 
109 
O70 
189 


2 CS ws SS 

CI -~) 
- am ay r 
nS oe 


~~] 


. 253 


. 104 
. 206 


. 156 
. 049 
. 074 
. 143 
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TABLE I 





CORRELATION COEFFICIENTS OF TEST SCORES AND COURSE GRADES—Continued 


Interest Questionnaire 
Science, Engineering, 
and Mathematics 


Social Studies 
English_ - - - 
Fine Arts 
Leadership - 
Humanitarian 
Sociable___- 
Withdrawal 
Reading 
Talking 


Botany 401 
L . 051 
D —. 191 
L . 023 
D —.119 
L . 193 
D —.145 
L . 054 
D —. 094 
L . 144 
D —.176 
L . 050 
D —. 094 
L .128 
D —. 183 
L . 086 
D —. 189 
L .114 
D —.214 
L . 238 
D . 125 
TABLE II 


History 403 


—. 184 
. 028 
. 030 
. 049 
. 038 
—.016 
—. 206 
. 070 
.011 
—.101 
. 032 
. 038 
—. 068 
. 094 
—.017 
.018 
—. 125 
—. 044 
—. 026 
—. 062 


MEANS AND STANDARD DEVIATIONS OF THE TEST SCORES OF THOSE WHO 
THOSE WHO Dip Not ELECT CERTAIN COURSES 


Botany 
401 
M o 
Verbal Intelligence __ 60.8 25.6 
Social Problems 
No. Sound Reasons 29.9 8.3 
Av. No. Sound 

Reasons 6° 4.3 
No. Inconsist. 

Reasons._.______ 18.1 14.7 
No. Democratic 

Conclusions 4.3 1.4 
No. Compromise 

Conclusions _ - 2.0 1.0 
Def. Spec. 

Privilege.___._._.. 3.5 3.4 
Def. Individualism 0.9 1.0 
Def. Dem. 

Principles__.... 9.6 3.8 
Def.Compromise_ 1.8 1.8 

Social Attitudes 
Liberalism 

Democracy___.. 65.0 10.3 

Nationalism____ 60.1 15.2 

Militarism_____ 61.0 13.6 

; ae _ 60.0 10.6 
Conservatism 

Democracy_.._._. 22.0 8.4 

Nationalism___. 23.8 12.3 

Militarism _ _ _ _- 25.6 10.5 

. ae 22.3 8.4 


Took Course 


History Fine Arts 
403 421 

M o M o 
53.8 29.0 65.1 22.9 
29.9 8.8 28.8 8.7 
3.8 1.2 4.0 1.3 
1.8 12.1 17.1 13:7 
4.4 1.4 4.1 1.5 
1.8 1.0 1.9 1.2 
3.8 3.5 4.0 3.3 
10 1.3 0.8 0.9 
10.1 4.4 9.7 4.2 
19 2.0 2.0 1.8 


63.3 9.6 63.2 9.3 
58.8 16.0 59.4 14.3 
59.3 15.9 63.2 12.3 
58.3 10.5 58.0 9.5 
23.0 9.4 23.0 7.5 
24.0 11.3 24.8 12.3 
26.0 12.5 24.4 10.6 
22.9 8.3 23.8 7.8 


Did Not Take 


Botany History 
401 403 
M o M o 
61.0 26.3 62.3 25. 
30.4 10.4 30.2 9 
4.0 1.3 4.0 1 
19.3 14.7 18.9 15 
4.5 1.5 4.4 1. 
19 1.1 2.0 1 
4.8 4.0 4.4 3. 
Le £2.30 i. +1. 
10.3 4.4 10.0 4. 
1.9 2.0 1.9 1. 
64.5 9.6 64.9 9. 
60.2 16.3 60.4 15. 
63.2 7.0 62.2 8. 
60.0 10.9 60.2 10. 
20.2 9.1 20.5 8. 
20.56 14.5 21.2 12. 
24.6 10.8 24.8 10. 
21.1 10.0 21.3 9. 
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—, 030 


997 
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. 301 
—.079 
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MEANS AND STANDARD DEVIATIONS OF THE TEST SCORES OF THOSE WHO Dip AND 


TABLE II 


COLLEGE FRESHMAN TESTING PROGRAM 


THOSE WHO Dip Not ELEcT CERTAIN CourRsesS—Continued 


Botany 
401 
M o 
Uncertainty 
Democracy -_- - -- 15.7 9 
Nationalism__.. 20.2 12 
Militarism - - - - - 16.4 10 
rae 19.8 10 
Consistency 
Democracy_.... 59.6 12. 
Nationalism__.. 55.3 18 
Militarism - - - -- 59.9 15. 
o” aa 59.4 10. 


Interpretation of Data 
General Accuracy 


Relative Values_ 53. 
Insufficient 
 Evidence_____ 50. 
Obviously TorF 59. 
Understatement 29. 
Beyond the 
Oe 43. 


intemporary Affairs 
Total Secore____.-- 


Part I, Pol.—Soe. 


Internat. Events 11. 
Total Score_____ 28. 
Part II, Cont. Culture 
BOOMS... ccncane Be 
Musicand Radio 9. 
Movies__...... 26 
Total Score__-__- 42. 
Interest 

Science, Eng’r., 
Math. _L 16 
D 10 
Social Studies. _L 19 
D 5. 
English. .____- L 20 
D 5. 
Fine Arts _L 19 
D = 3z. 
Leadership____L 8 
D 3. 

Humanitarian- 
ism ees 12. 
2. 
Sociable. L380. 
> 
Theoretical.___L 9. 
D 4. 
Reading _._.L $4. 
: D9. 
Talking .. L 18. 
D 6. 


onorn 
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CORMPOADWDAAD OCOWDCONNAAS 


Took Course 


History Fine Arts 
403 421 
M o M o 
15.7 11.3 15.6 8.1 
21.1 15.5 21.5 14.5 
18.2 13.7 16.1 10.6 
20.7 13.0 21.3 10.7 
59.6 12.5 58.8 12.1 
55.0 16.8 54.0 17.3 
58.0 15.0 58.8 14.5 
58.9 10.3 57.2 10.4 
50.3 14.1 50.9 11.8 
45.0 19.2 45.4 15.6 
57.1 14.6 58.9 14.4 
28.9 11.0 29.1 9.7 
45.2 15.8 46.7 12.9 
70.0 28.7 69.8 27.2 
12.1 5.5 11.0 5.7 
27.6 15.4 25.8 15.8 
14.3 6.0 14.0 4.7 
8.4 3.0 8.8 3.6 
26.2 7.8 27.6 8.8 
41.4 16.8 44.6 17.3 
16.7 9.8 17.5 9.6 
11.6 9.3 10.0 8.0 
20.4 8.0 17.0 8.0 
5.9 5.1 6.0 5.7 
20.4 8.0 19.5 7.9 
6.4 5.7 5.7 65.9 
19.4 5.7 24.2 4.0 
4.7 3.8 2.3 2.1 
9.1 3.9 7.7 38.7 
3.3 2.8 3.4 2.9 
13.8 4.8 11.6 5.0 
2.5 2.6 2.8 3.1 
30.9 7.1 29.4 6.8 
3.2 2.8 3.6 3.2 
9.5 6.0 9.1 6.1 
4.7 3.8 4.3 3.9 
33.8 11.2 32.7 13.0 
10.9 8.2 9.8 8.8 
18.2 5.9 18.0 6.1 
7.0 65.2 6.2 6.5 
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Did Not Take Course 
Botany History Fine Arts 
401 403 421 

M o M o M ¢ 
18.5 8.6 17.8 8.7 17.6 9.8 
23.2 16.6 21.6 13.6 21.5 13.8 
.8 21.8 27.4 11.1 WT MT 
21.4 11.8 20.9 11.1 20.8 11.5 
59.0 12.7 59.1 12.5 69.3 12.6 
47.4 22.7 49.3 22.3 49.6 22.0 
41.2 28.4 45.9 27.4 46.0 27.3 
58.5 11.0 58.8 11.0 59.0 11.0 
52.3 12.4 53.3 11.5 62.8 11.9 
46.9 17.0 48.6 16.2 48.5 16.9 
59.4 14.2 659.8 14.0 59.5 14.0 
26.5 9.6 41.8 9.7 27.3 10.0 
45.0 13.8 42.2 12.9 44.0 13.4 
72.0 30.8 71.6 30.5 71.6 30.6 
12.8 6.2 12.6 6.1 12.8 6.0 
30.7 17.0 31.2 17.1 30.4 11.9 
14.3 5.9 14.3 5.6 14.3 5.8 
8.7 3.4 89 3.4 8.8 3.3 
24.3 10.2 25.7 9.0 25.5 8.8 
40.6 18.6 41.0 18.5 40.5 18.3 
19.6 10.6 18.8 10.3 18.7 10.4 
8.9 7.9 9.1 7.8 9.4 8.8 
19.4 8.3 19.0 8.3 19.6 8.0 
5.4 56.3 5.4 538 65.4 65.2 
19.3 8.5 19.4 8.5 19.6 8.5 
6.2 5.6 6.0 5.7 6.0 65.8 
19.1 6.2 19.2 6.2 18.5 6.0 
4.1 3.7 3.9 3.7 4.3 3.9 
3.5 4.3 8.4 4.5 8.7 4.3 
3.7 3.0 3.2 3.0 3.1 3.0 
12.8 5.6 12.6 4.9 13.0 4.8 
2.8 2.6 2.3 2.7 2.2 2.6 
29.7 7.4 29.8 7.2 30.1 7.1 
3.3 3.4 3.2 3.2 3.1 3.2 
10.7 5.2 10.5 6.2 10.4 6.2 
3.8 3.4 3.9 3.6 3.9 3.6 
34.0 12.1 34.1 12.0 34.3 11.6 
95 9.0 9.4 8.4 10.0 8.0 
19.0 7.0 19.0 6.9 19.0 6.9 
6.0 5.1 5.9 5.0 65.2 5.0 








aval 


4 
" 


ee ee Ee Oe eee 


De pear od ig EC WTS a AA Bc iia 





288 JOURNAL OF EXPERIMENTAL EDUCATION 


Psychological Test scores of Fine Arts stu- 
dents, in Table II, reveals that this assump- 
tion is not justified. 


SoctAL PROBLEMS TEST 


The correlation between Botany grades and 
scores on Total Number of Sound Reasons is 
0.273, and with Average Number of Sound 
Reasons per Conclusion is 0.230. The cor- 
relation between History grades and scores on 
Average Number of Sound Reasons per Con- 
clusion is 0.424, and with Number of Incon- 
sistent Reasons is — 0.362. While scores of 
reasoning skills correlate to some extent with 
History grades, correlations with scores of 
“point of view variables” such as Democratic 
Conclusions, or Compromise Conclusions are 
good approximations to zero. It might seem 
that in any consideration of social problems, 
point of view should be a concomitant of 
reasoning. 


SoctAt ATTITUDES TEST 


Significant positive relationships exist be- 
tween grades in both Botany and History and 
scores indicating liberalism. Possibly the bet- 
ter students score higher on liberalism because 
they are more capable thinkers; perhaps they 
are more easily indoctrinated. 


Of interest is the correlation coefficient, 
0.352, of Fine Arts grades and scores on lib- 
eral attitude toward nationalism. This may 
be a chance relationship such as was obtained 
between the hardness of asphalt pavement, 
over a period of years, and the salaries of 
school superintendents. Perhaps there is a 
more plausible explanation. Much of the 
recognized art originated in Europe. The 
fact that it is recognized, accepted, and in 
many cases emulated, would seem to indicate 
that the better art students are not averse 
to accepting foreign ideas, particularly if they 
pertain to Art. Since statements in the na- 
tionalism sub-test are based upon the ac- 
ceptance or rejection of “foreign” rather than 
“idea”, it does not seem illogical to assume 
that the better beginning art students will 
tend to interpret these statements in terms 
of art. 


The scores on consistency of attitudes 
show significant relationships with grades in 
all three courses. 
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INTERPRETATION OF Data Test 


The correlation of the score on Relatiy: 
Values with History grades is 0.584, and with 
Botany grades, 0.407. These relationships 
are noticeable both because of their size rels. 
tive to the correlations of other test scores 
with course grades, and because the Botany 
Department stresses interpretation of data as 
a teaching objective. 

History grades correlate more highly wits 
scores on ability to recognize insufficient ey. 
dence; likewise with the scores on ability ; 
recognize probably true or probably {als 
items. Botany grades correlate more high) 


with the scores on ability to recognize try: 
and false statements. 


CONTEMPORARY AFFAIRS TESI 


The highest correlations of the Conten 
rary Affairs variables are with Histor 
grades. The total scores and the Part | 
Political-Social score, have correlations 
0.306 and 0.330, respectively. Chronolog 
excepted, the content of History 40; 
similar to the content of the Contemporary 
Affairs Test. Thus it is not unreasonable 


do well in the other. 

For Botany, the Political-Social scor 
the score on International Events have sig- 
nificant correlations. It may be observed the! 
both of these correlations are higher than for 
Total Score. 

Noticeable is the absence of significant re- 
lationships between course grades and scores 
on Contemporary Culture, although there is 4 
positive correlation between History graces 
and scores on this sub-test. 


INTEREST QUESTIONNAIRI 


Botany grades show six significant corre. 
tions with Interest Questionnaire scores. Dis 
like of Science, Engineering, and Mathema'- 
ics, —o.91; Like of English, 0.193; Dis 
like of Reading, —o.214; Dislike of S~ 
ciable Interests, — 0.183; Dislike of Wit 
drawal Interests, — 0.183, and Like of Tals- 
ing and Conversation, 0.238. In such a (- 
lection of correlations, a tolerance of, rathe! 
than an interest in Science, Engineering, 40 
Mathematics might be implied. 

The absence of a high correlation between 
History grades and scores showing an inter 
est in the Social Studies, together with the 
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) lack of positive relationship of Botany grades 
- to scores on Interest in Science, Engineering, 
" and Mathematics, might raise a question as 
- to whether, under present conditions, interest 


in success is not more pertinent to good 


 orades in an academic subject than is inter- 
" est in the subject, per se. Further, do teach- 
- ers consider interest to be a device for raising 
' the achievement level of a group, rather than 


, concomitant of good teaching? It can be 
concluded that interest, as measured by the 
Progressive Education Association question- 
naire, is not absolutely essential to success in 
Botany 401 and History 403. 

The correlation of Fine Arts grades and 
scores on Interest in Fine Arts is 0.301. There 
is no evidence to indicate whether interest is 
, factor in success or success is a factor in 
nterest. 

It may be noted that most of the “Like” 
correlations, involving social interests, are 
positive, and correspondingly the “Dislike” 
orrelations are negative. These relationships 
illustrate the fallacy of a popular conception 
that a student’s “sociability” is in inverse 
proportion to his scholarship. From the 
standpoint of counseling, the reverse is 
worthy of consideration. 

The most significant relationships indicated 
by this study are between Botany and the 
0. 8. U. Psychological Test, r = 0.526; His- 
tory and the O. S. U. Psychological Test, 
r= 0.581; Botany and the Interpretation of 
Data Test, r=-0.407, and History and the 
Interpretation of Data Test, r 0.584. No 
attempt has been made to select a battery 
of tests most predictive of marks in each 
course. Statistically, the best tests were the 
0. 5. U. Psychological Test and the Inter- 
pretation of Data Test. 


IX. A STUDY OF CHARACTERISTICS 
OF EDUCATION FRESHMEN 
WHO ENTERED OHIO 
STATE UNIVERSITY 
IN 1938 


Wape S. AmstutTz 


_It is proposed in this study to investigate 
‘he relationships between various test scores 
and the grades received in the freshman 
courses. It is recognized that college marks 
are not the sole desirable evidence of satis- 
‘actory progress toward adequate profes- 
‘ional and personal adjustment. Such marks 
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constitute only one of the possible criteria. It 
is hoped that this study, limited to a specific 
problem, will be the starting point for further 
studies involving other criteria, not now 
available. 

The five tests listed in the introduction to 
this series of articles, namely the Cooperative 
Contemporary Affairs Test, an Interpretation 
of Data Test, an Interest Questionnaire, a 
Social Problems Test, and a Social Attitudes 
Scale, as well as the Ohio State University 
Psychological Examination constitute the 
battery of tests which are the basis for this 
study. 

The point-hour ratio for the autumn quar- 
ter of 1938 was selected as the criterion 
variable. An average grade was obtained by 
taking the ratio of points to hours carried. 
For each hour of grade A, four points were 
given; for each hour of B, three points; for 
each hour of C, two points; for each hour of 
D, one point; and for each hour of E 
(failure), no point. Before one can well go 
ahead with the development of new criteria 
for judging academic performance and pro- 
fessional development, those criteria at hand 
should be examined. This study has as its 
problem, the investigation of the relationship 
between a traditional criterion of academic 
success and the various test scores obtained. 
The next step, not here attempted, is to con- 
struct other useful and valid criteria which 
are fairly independent of the course marks. 

The Wherry—Doolittle Test Selection 
Method’ has been employed to find the maxi- 
mum multiple correlation with the criterion 
after a correction is made for the chance 
error for each test added. The variables or 
tests are selected in the order of their con- 
tribution to the multiple correlation. The 
first test to be selected is that test which has 
the greatest correlation with the criterion. 
The second is the one which in combination 
with the already selected test gives the great- 
est multiple correlation. The third test is 
that one which combined wih the two se- 
lected raises the multiple correlation the 
most, etc. The increase in the multiple be- 
comes less and less while at the same time 
the chance error increases. The selection may 
continue until the addition of another test 
adds more chance error than actual validity 
or until all tests are included. The Wherry 
shrinkage formula indicates when the point of 


1 Designed by Dr. R. W. Wherry, University of North 
Carolina. 
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adding more chance error than validity is 
reached so that no further addition of tests 
is feasible. 

The work is begun on each test at the same 
time and carried out simultaneously with the 
selection of one test at the end of each step. 
This method holds only if no two tests have 
any part in common, which eliminates 
spurious correlations. For instance, if one 
variable were the total of Part I and Part II, 
the variable containing the total may be in- 
cluded if Part I and Part II are excluded 
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from the study; or, if both Part I and Patt u 
the total must be excluded. Such exclusi ae 
were used at the appropriate points jn the 
calculations. For example, when variable + 
Part I of the Contemporary Affairs, entered 
the selected test battery, variable 37, T, tal 
Score for the Contemporary Affairs, w,; 
dropped from further consideration. 
Table I presents the zero order correlations 
of each of the test scores with the criterion 
These range from 0.012 for variable 95 o 


TABLE I 


CORRELATION OF VARIABLES WITH CRITERION 


Coefficient of 
Variable Correlation 
with Criterion 
A. Interpretation of Data Test 
ae 


Coefficient of 
Variable Correlation 
with Criterion 
60. Average number of sound 
reasons per conclusion. . 265 
62. Number of reasons incon- 
sistent with course of 


66. Number of democratic 
conclusions__________- 067 
68. Number of compromise 
conclusions._______- 073 
70. Defense of special 
privilege____________. 067 
73. Defense of individualism 146 
74. Defense of democratic 
principles___________- 150 
75. Defense of compromise 
for sake of expediency - 094 


E. Interest Questionnaire 
76. Like Science, Engineer- 


ing, Mathematics_ ___- 069 

77. Dislike Science, Engi- 
neering, Mathematics - 179 
78. Like Social Studies_____- 016 
79. Dislike Social Studies___- 119 
80. Like English________- 165 
81. Dislike English. _____-_-- . 240 
86. Like Fine Arts_________- 092 
87. Dislike Fine Arts_-_-_-_-- 189 
94. Like Leadership __- --- : 024 
95. Dislike Leadership - ----- 133 
96. Like Humanitarian -_---_- 012 
97. Dislike Humanitarian --_- 132 
98. Like Sociable__________- 018 
99. Dislike Sociable________- —. 131 
100. Like Theoretical_______- 113 
101. Dislike Theoretical. _-- 188 
102. Like Reading_________-- 11% 
103. Dislike Reading -- - ---- 239 
108. Like Talking. ________-- 077 
109. Dislike Talking_-__------ 100 


F. Intelligence Test 
118. Ohio State University se 
Psychological Test - - - - 607 


G. High School Record 


General Accuracy ______- .494 
3. Insufficient Data_______- .318 
4. Accuracy with true and 
false statements. _____ . 443 
6. Overcautiousness__ _ _ - _- —. 199 
7. Beyond the facts___----- —. 305 
B. “—_ of Beliefs 
9. Liberalism in democracy . 280 
13. Liberalism in nationalism . 344 
14. Liberalism in militarism . 361 
15. Total liberalism ___-__-- . 300 
: 16. Conservatism in 
democracy __________- —. 225 
20. Conservatism in 
i nationalism _______- —, 296 
21. Conservatism in 
i militarism _____-- - - - : —. 336 
22. Total conservatism” _-_-_- —.274 
; 23. Uncertainty in democracy —. 081 
t 27. Uncertainty in 
” nationalism. ________- —.115 
) 28. Uncertainty in 
' militarism. ________- —. 088 
i 29. Total uncertainty -___- - —. 070 
: 30. Consistency in 
i democracy ____------- . 303 
; 34. Consistency in 
4 nationalism - - - ----_-- .318 
35. Consistency in 
militarism ___________- . 361 
36. Total consistency - Pe nou . 402 
C. Cooperative Contemporary 
Affairs Test 
37. Total Test____.____-_- . .317 
38. Political-Social._______- . 302 
44. Knowledge of Inter- 
national Events__-___-_- .177 
| 50. Contemporary Culture . 259 
: Gl. BOGME.........-...-«- z .178 
: 53. Music and Radio. ___- . 146 
a] 54. Movies__._._..______- = . 158 
; D. Social Problems Test 
io 59. Number of sound reasons 





checked___________- . 


. 223 


119. Rank in high school : 
graduating class_- ---- 441 
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‘re. to 0.007 for variable 118 on the Ohio 
State University Psychological Examination. 
Other than the Ohio State University Psy- 

Examination, the Interpretation of 
t seems, in general, to have the great- 
ity, for predicting academic success. 
y be “somewhat to be expected” 

the test apparently reflects many of the 
emanded in class work. 

fable IL indicates the variables selected by 

Wherry—Doolittle Method in the order 
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ing quarter. If such a figure be taken as an 
estimate of the reliability of the criterion, 
then the battery of variables selected ap- 
proaches the limit of correlation with the 
criterion. 

The multiple correlation of 0.692 indicates 
that approximately one half of the variance 
for predicting academic success has been ex- 
plained. The addition of the last test ac- 
counted for less than o.ooo1 of a perfect re- 
lation between the criterion and the tests pre- 
dicting it. No further test additions are 


TABLE II 


BETA AND b WEIGHTS, CORRELATION AND CUMULATIVE MULTIPLE CORRELATION 
COEFFICIENTS FoR EACH SELECTED VARIABLE 


Selected Variable 


Ohio State University 
Psychological Test - 
General Accuracy 
Interpretation of Data Test 
119. Rank in High School 
Graduating Class 
§. Total Consistency 
Seale of Beliefs) 
<7, Dislike Fine Arts 
Interest Questionnaire) _ __ 
38. Political-Social 
Cooperative Contemporary Affairs 
), Average Number of Sound Reasons per 
Conclusion (Social Problems Test 
Contemporary Culture 
Cooperative Contemporary Affairs 


i their selection. This table also shows the 
multiple correlation coefficient (corrected for 
chance error) at the stage represented by the 
inclusion of that test in the battery. For 
example, the addition of the second test, vari- 
able 1 on General Accuracy (Interpretation 
ot Data Test) raised the correlation with the 
criterion from 0.607 to 0.649. The addition 





Beta b Weight 


Cumulative 
Coefficient of 
Multiple 


Coefficient of 
Correlation 


Weight with Criterion Correlation 

. 325 009 607 607 
. 198 . 009 494 649 
. 181 . 096 .441 668 
. 120 . 008 . 402 678 
.114 . 031 . 189 684 
.111 . 005 . 302 . 689 

070 004 . 265 . 692 
.041 . 002 . 259 . 692 


feasible since the tests are selected in order 
of their contribution to the multiple. 

The Table also shows the Beta or net 
regression coefficients and the b or gross score 
regression coefficients. The Beta weights are 
used to predict the individual’s criterion 
score only when his test scores have been 
transformed into standard scores. The regres- 
sion equation is 


be = 325X114 + .198X, + .181X,19 + .12OXgg — .IT4Xg7 + .IIIXy, + .070X,, — .041X 56 





of a third variable, 119, Rank in High 
School Graduating Class, raised the multiple 
correlation to 0.668, etc. All the eight vari- 
ables selected produced a multiple correlation 
0! 0.692 (corrected for chance error) with 
point-hour ratio. This is about the same mag- 
nitude as the correlation of the point-hour 
ratio of one quarter with that of the succeed- 





By Beta weights only are contributions of 
the various tests on a comparable basis. They 
tell us what part of a standard score should 
be included to give the most probable predic- 
tion. Thus by examination of the Beta 
weights we learn the relative importance of 
each selected test toward the purpose in 
question. 
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TABLE III 
INTERCORRELATION OF SELECTED VARIABLES 


Selected Variable 118 1 
118. Ohio State University 
Psychological Test__- 47 
1. General Accuracy 
(Interpretation of Data) 477 
119. Rank in High School 
Graduating Class_____-- . 448 . 315 
36. Total Consistency 
(Seale of Beliefs) _ _ - - . 381 . 400 
87. Dislike Fine Arts 
(Interest Questionnaire -.179 . 025 
38. Political-Social 
(Contemporary Affairs) . 320 . 230 
60. Average Number of 
Sound Reasons Per 
Conclusion 
(Social Problems Test) - - . 308 
50. Contemporary Culture 
(Contemporary Affairs) .418 


bo 
oS 
or 


_ 
+) 
@o 


In the practical problem of setting up pre- 
dicted scores it is more convenient to use 
gross scores regression coefficients. These are 
usually referred to as b weights. With b 
weights the regression becomes 


s * .009X,,, + .0ogX, + .096X,,, + .008X,, — 


+ .005X,;, + .004X,,, — .002X, + K 


The intercorrelations of the selected vari- 
ables are shown in Table III. 


SUMMARY 


In the comparison of the Beta weights we 
find that the Ohio State University Psy- 
chological Examination has the largest. It is 
interesting to note that Dislike for Fine Arts 
and Contemporary Culture have negative 
weights. 

The prediction battery is 

118. Ohio State University Psychological 


Test 
1. General Accuracy (Interpretation of 
Data Test) 
119. Rank in High School Graduating 
Class 


V 
119 


60. 


50. 


\ V \ \ 
36 37 38 60 
. 381 .179 . 320 . 308 {) 
.400 —.025 . 230 . 205 
. 222 —.055 .141 . 164 i4 
~. 066 .224 174 146 
066 011 023 
. 224 .011 114 4. 
.174 . 023 .114 
. 146 . 135 , 481 107 


. Total Consistency (Scale of Beliefs) 
. Dislike Fine Arts (Interest Question- 


naire) 


. Political-Social (Contemporary Af- 


fairs Test) 


.031X,, 


Average Number of Sound Reasons 
per Conclusion (Social Problems 
Test) 


Contemporary Culture (Contempo- 
rary Affairs Test) 


It is significant to note that scores from 
each of the tests used appear among the se- 


lected 


tests. This perhaps attests the excel- 


lence of the a priori selection of tests to be 
used in this program. The significant pre- 
dictors are, with the exception of Dislike for 
Fine Arts, scores which reflect abilities, in- 
formation or consistency of response, rather 
than a point of view such as liberalism, 
nationalism, or conservatism. 
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PERFORMANCE IN THE IOWA QUALIFYING EXAMINATION 
OF MAJORS IN VARIOUS ACADEMIC DEPARTMENTS 
WITH IMPLICATIONS FOR COUNSELING 


Dewey B. Stuit and Mary CARROLL DONNELLY 
State University of lowa 


The choice of a major is a problem which 
troubles a rather large number of college stu- 
dents. Because of superficial interest in a sub- 
ect, admiration for a certain instructor, or 
the suggestion of a friend, a student may 
make a choice which he will regret later in 
his educational career. Many students do not 
have a satisfactory opportunity to evaluate 
their achievements in an objective way and 
‘hrough that means arrive at a sound judg- 
ment concerning their field of concentration. 
It has been suggested by many writers that 
aptitude and achievement tests should provide 
the student with the necessary objective facts. 
However, the vast majority of studies in 
which such tests have been used have not 
oroved exceedingly helpful because the results 
ire confined to correlations showing the rela- 
tionship between a predictive index and a 
criterion. Generally the differential prediction 
value of various measuring instruments has 
not been ascertained. 

The purpose of the present study is to 
investigate the possibility of utilizing test 
results in that area of counseling which in- 
volves the student’s choice of a major subject. 


| Specifically, it seeks to determine whether 


there are any differential characteristics of 
majors in various academic departments 
which are revealed by their performance in 
the lowa Qualifying Examination. This ex- 
imination, consisting of the Iowa High School 
Content Examination, the Mathematics Apti- 
tude Test and English Training Test of the 
lowa Placement Examinations, and the Iowa 
Silent Reading Test, is administered to all 
‘reshmen when they enroll in the University. 
The purpose of this examination is to provide 
some indication of the student’s scholastic 
aptitude and to assist the counselor in his 
educational advisory work with students, par- 
ticularly at registration time. The present 
‘tudy investigates the initial test performance 
| students who later succeed in obtaining 
scademic degrees in various departments. If 
ispection reveals that one may partially de- 
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scribe the successful students in various aca- 
demic departments in terms of their variabil- 
ity of achievement in the different tests of the 
Iowa Qualifying Examination, such informa- 
tion should afford valuable supplementary 
material in counseling a student concerning 
his choice of a major subject. If it is found 
that there are no such differential character- 
istics, then the use of the test profile, as such, 
is of little value when counseling in this area. 
In brief, the questions which this study seeks 
to answer are as follows: 


(1) With respect to initial test scores, is 
there a characteristic test profile for stu- 
dents who have successfully received aca- 
demic degrees in a given department? 

(2) Is there a characteristic lower limit 
suggestive of a critical score for majors in 
various academic departments? 


Nine major departments in the College of 
Liberal Arts at the State University of Iowa 
were selected for study. The selection in- 
cluded: (1) Mathematics; (2) English; 
(3) Journalism; (4) Political Science: 
(5) History; (6) Fine Arts: Music and 
Graphic Arts; (7) Physical Science: Chem- 
istry and Physics; (8) Biological Science: 
Biology and Botany; (9) General Science. 
These groups were selected in an attempt to 
represent all of the major fields of concentra- 
tion, combining those fields assumed to be 
closely allied. 

In order to secure an adequate sample of 
students in each of these departments, it was 
necessary to obtain data from classes gradu- 
ated in June Convocations at the State Uni- 
versity of lowa for the years 1935-1938 in- 
clusive. All students graduated during that 
period who had taken the Iowa Qualifying 
Examination upon entrance to the University 
were included as subjects for the study. This 
eliminated students who had transferred to 
the University from other institutions and 
those who had entered during the summer 
terms. The total group included 365 students. 
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Scores in the individual examinations and the 
composite, consisting of the sum of the raw 
scores of the four individual examinations, 
were converted to percentile ratings. 

The results of this study provide a quanti- 
tative description of the performance of the 
nine major groups in the individual examina- 
tions and composite score of the Iowa Qual- 
ifying Examination. Detailed findings are 
summarized in Tables I and II. 

Table I shows the means and standard 
deviations of the major groups in each of the 
individual examinations and composite score 
of the Iowa Qualifying Examination. It is 
interesting to note that the two highest rank- 
ing groups in terms of composite score, Math- 
ematics and English, are also the least vari- 
able. The third ranking group, Physical 
Science, presents an interesting contrast in 
that it appears more variable than all other 
groups except the Fine Arts Majors, who 
show the widest variability of any group. The 
lowest ranking groups, Biological Science, 
Fine Arts, and Political Science all show wide 
variability. 

The examinations in which each of the 
major groups receives its highest and lowest 
rankings constitute interesting observational 
data. Mathematics, Physical Science, and 
General Science Majors show their highest 
mean score in the Mathematics Aptitude 
Examination; English and Fine Arts Majors 
in the English Training Examination: Bio- 
logical Science, Political Science, and History 
Majors in the High School Content Exam- 


ination; and Journalism Majors in the | 
Silent Reading Test. 

With respect to low scores, it will be note: 
that the Mathematics Majors earn: 
lowest mean score in the Iowa Silent Reais» 
Test; Physical Science, General Scien: 
Biological Science, and Political Scie 
Majors in the English Training Exa tior 
English and History Majors in the \o 
ematics Aptitude Examination and | 
and Fine Arts Majors in the Mathemotix 
Aptitude and High School Content Fy. 
ination. 

An additional representation of va: 
in performance is given in Table I] 
shows the decile ratings of all major 
in the individual examinations and < 
score of the Iowa Qualifying Examina‘ 
per cent of cases. With regard to th 
limits, it will be noted that the grou; 
ing highest per cents in the roth decile ar 
the following: Mathematics Majors in th 
Mathematics Aptitude Examination, Matb- 
ematics Majors in composite score, Math. 
ematics Majors in High School Content F: 
amination, English Majors in English Train- 
ing Examination and English Majors i: 
posite score. In these instances, 40° or mor 
of the cases fall within the roth decil: 


W3 


Among the groups showing the lowest per 


cents in the roth decile are History Major 
in the English Training Examination, Jou- 
nalism Majors in the High Schoo! Content 
Examination, Biological Science Majors in 
the Iowa Silent Reading Test and Fngiish 


TABLE I 


MEAN PERCENTILES AND STANDARD DEVIATIONS OF THE MAJoR GROUPS IN EACH OF THE INDI 
VIDUAL EXAMINATIONS AND COMPOSITE SCORE OF THE IOWA QUALIFYING EXAMINATION 


H.S.C. 
Major - N M o 


Mathematics _______________ 21 82 20.0 
 —- Sa 77 18.2 
a 72 27.0 

ee 67 28.1 


Phys. Science — 
Gen. Science ____ 


a 61 25.3 
History _____~ ee 67 23.4 
eae 75 64 28.0 
EN a a 47 54 30.0 


ae 62 26.5 


1.S.R.* M.A. E.T. C.S 
M a M Co M og M ¢ 
73 22.6 938 5.3 75 25.8 86 21.4 
77 22.0 66 25.3 82 18.1 78 21.2 
67 27.1 76 25.0 66 30.0 73 28.4 
61 25.2 71 23.7 55 26.8 69 23.6 
68 25.1 61 26.9 67 23.5 66 23.6 
63 30.3 55 299 62 252 63 284 
59 26.3 59 28.7 56 27.1 61 27.9 
60 22.4 54 31.4 70 26.3 60 30. 
60 26.9 59 26.2 55 24.2 59 27 


The abbreviations are to be interpreted as follows: 


H.S.C. = High School Content Exami- 


nation 
1.S.R. = Iowa Silent Reading Test 


M.A. = Mathematics Aptitude Exam 

ination 
E.T.=English Training Examination 
C.S. = Composite Score 


* Iowa Silent Reading Test Scores were not available for one English, one Political Science, 


and three Fine Arts Majors. 


H 


~ 
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TABLE II 


RaTINGS OF ALL MAJoR Groups IN INDIVIDUAL EXAMINATIONS AND COMPOSITE SCORE OF 
THE IOWA QUALIFYING EXAMINATION IN PER CENT OF CASES 


1 2 3 4 5 6 7 8 9 10 N 
WS eer 4.8 1.8 14.3 4.8 19.0 2.4 21 
Math. LS.R. _ 4.8 4.8 1.8 18 143 143 286 238 21 
ne WA £8 233 Ti4 23 
E1 1.8 14.3 1.8 14.8 19.0 12.8 21 
cS 4.8 4.8 48 19.0 66.7 21 
L. 
S( 3.4 17 8611.9 3 6.8 138.6 271 322 59 
a £7 Ly 6.8 10.2 10.2 8.5 yn Ry 9.6 8 
TA e. 5 8.5 8.5 11.9 6.8 15.3 25.4 16.9 ie) 
"1 1.7 6.8 6.8 10.2 8.5 18.6 47.5 9 
1.7 5.1 186 119 8.5 18.6 40.7 59 
rnvs. Se. 
H.S.C : 9.0 4.5 15 182 1.5 45 182 364 22 
tl: See 4.5 4.5 13.6 18.2 13.6 9.0 4.5 31.8 22 
M.A 9.0 4.5 9.0 1.5 9.0 27.3 16.4 22 
P E.7 1.5 13.6 3.6 4.5 1.5 18.2 9.0 1.8 22 
9.0 9.0 9.0 9.0 9.0 18.2 36.4 22 
M 
Vat , 
nt I HS. : 7.4 7.4 11.1 7.4 7.4 7.4 25.9 259 27 
Tea LS.R 3.7 11.1 18.5 22.2 3.7 «= «18.5 3.7 18.5 7 
se MLA, ; 3.7 7.4 18.5 148 11.1 111 333 27 
1 E.T. me 7 3.7 11.1 18.5 111 148 111 3.7 148 27 
CS. me 7.4 8.7 86185 48111 148 18.5 222 27 
=? LS.C. = 54 89 10.7 89 10.7 89 160 196 10.7 56 
Ma} LS.R : - a 3.6 3.6 7.1 5.4 8.9 143 10.7 286 16.0 56 
J MLA ey 5.4 5.4 1438 7.1 54 19.6 10.7 10.7 19.6 56 
vnten E.1 ee ee 3.6 7.1 16.0 125 143 8.9 125 25.0 56 
ea _ Sean 8.9 12.5 8.9 1.7 160 1438 17.9 19.6 56 
English Hist 
H.S. aan 5.7 2.9 5.7 5.7 143 171 114 171 200 85 
L.S.R. a 8.6 2.9 5.7 11.4 5.7 114 148 17.1 20.0 35 
M.A. eanacese a 5.7 148 148 5.7 5.7 11.4 29 229 143 35 
I) E.T. whe 5.7 5.7 11.4 5.7 5.7 200 200 17.1 8.6 35 
10 CS. situa 5.7 5.7 5.7 143 20.0 114 114 229 35 
” ~ Pol. Se. 
ie H.S.C, nm 48 5.3 6.7 8.0 4.0) 2.7 14.7 18.7 16.0 20.0 15 
. LS.R. oe ae 5.3 5.3 16.0 2.7 160 133 13.3 93 14.7 14 
8 2h | aaa 5.3 4.0 93 80 13.3 5.3 10.7 12.0 14.77 173 15 
Lt op eee 1.3 6.7 8.0 14.7 120 93 10.7 12.0 10.7 183 75 
SORE sa Aitanneninsians -40 67 53 66 10.7 =80 183 160 1838 160 75 
¥ 28.4 F, Arts 
1 i H.S.C, ..... 85 10.6 43 14.9 6.4 10.6 64 106 106 17.0 47 
yo | es 45 23 68 159 182 23 45 136 68 25.0 44 
19 | Sees 4.3 8.5 21.3 4.3 6.4 64 149 21 106 213 47 
aa 43 64 21 106 85 149 85 85 362 47 
Exam om --------- 6.4 21 106 106 17.0 2.1 43 106 128 28.4 47 
Bio. Se. 
nination H.S.C. 


Science, E T. ¢ ea------ 4.3 
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Training Examination and Political Science 
Majors in the English Training Examination. 
In these instances, 14% or less of the cases 
fall within the roth decile. It is noticeable 
that the examination in which these lower 
scoring groups most frequently fail to score 
high is the English Training Examination. 
Fine Arts Majors are an exception, earning 
their highest score in the English Training 
Examination. 

Probably the chief interest in this table lies 
in the possibility of establishing lower limits 
or “critical scores” for the major groups in 
the individual examinations. It is recognized 
that the establishment of such “critical 
scores” is an arbitrary matter. While it is 
impossible to say that cases rating lower in 
the examinations than the groups studied 
would not have succeeded, it is possible to 
express quantitatively the relative achieve- 
ment of successful groups in this study. For 
example, the most striking observation is that 
with respect to Mathematics Majors, less than 
five per cent fall below the ninth decile in the 
Mathematics Aptitude Examination. In the 
case of English Majors, only slightly more 
than 25% fall below the eighth decile in the 
English Training Examination; less than two 
per cent fall below the fifth decile. 

Such data as these may be contrasted to 
that of the Fine Arts Majors, for which on 
three of four tests, approximately 45% fall 
within and below the fifth decile. In a similar 
manner, by adding up from the bottom one 
can determine the per cent of cases who fall 
at or below a certain decile rating in each of 
the examinations. For example, if one were 
interested in determining the percentage of 
cases of Mathematics Majors falling below 
the fiftieth percentile in any examination this 
could be done by adding all the percentages 
falling below the sixth decile for that exam- 
ination in Table II. 

In order to make a more careful statistical 
study of the results, analysis of variance’ was 
employed in studying the significance of dif- 


1 All of the actual computations are not reproduced here 
because of limitations of space. The means, standard devia- 
tions and differences in means can be obtained from Table I 
A discussion of the statistical technique known as Analysis of 
Variance may be found in: 

Fisher, R. A.: Statistical Methods of Research Workers. 
Sixth Edition, Oliver and Boyd, Edinburgh, 1936, pp. 214-290. 

Lindquist, E. F.: Statistical Analysis in Educational Re- 
search. Houghton—Mifflin Company, New York, 1940. In 
Publication. 

Snedecor, G. W.: Statistical Methods, Collegiate Press, Inc., 
Ames, Iowa, 1938, pp. 179-247. 

Vule. G. U.; and Kendall, M. G.: An Introduction to the 
Theory of Statistics. Eleventh Edition, Charles Griffin and 
Co., London, 1937, pp. 444-458. 


-~] 
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TABLE III 
RESULTS OF ANALYSIS OF VARIANCE Wyty, 


d.f. 


. Mathematics Majors 


Between tests__ 4 
Within tests___ 100 


a 104 
F Ratio 3.53* 


. English Majors 


Between tests__ 4 
Within tests___ 289 
TOM ocnna — 
F Ratio 4.78** 


Between tests__ 4 
Within tests___ 105 


IE sistance 109 


. General Science Majors 


Between tests._. 4 
Within tests___ 130 


a 134 
F Ratio 1.76 


. Journalism Majors 


Between tests__ 4 
Within tests_.__ 275 


. see 27 
F Ratio 1.58 


. History Majors 


Between tests__ 4 
Within tests_.__ 170 
eT 174 


. Political Science Majors 


Between tests__ 4 
Within tests___ 369 


TESTS AND BETWEEN TEST MEANS 
VARIOUS MAJoR GROUPS 


Sum of 
Squares 


5611.4 
39769.5 


8187.4 
123636.5 


3. Physical Science Majors 


1599.5 
75033.0 


4443.4 
81638.5 


2621.1 
113733.1 


2602.5 


121831.3 


7891.1 
194530.0 


570.66 


80042.53 


a 37: 

. Fine Arts Majors 
Between tests__ 4 
Within tests_.__ 227 
a 231 
F Ratio 2.30 

. Biological Science Majors 
Between tests.__ 4 
Within tests___ 110 
i reee 114 


FOR THE 


Variar ce 


1402.8 


ooTF 


PU46 8 


4278 


415 


19 fF 
42.00 
mor 65 


* To be significant at the 1% level “F” must 
equal 3.51 and at the 5% level 2.46. P 
** To be significant at the 1% level ‘f 
must equal 3.38 and at the 5% level 2.40. 
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14028 


PARR 


399.8 


6552 


415.5 


TABLE IV 


sumMarRY OF DIFFERENCES IN MEANS AND 
Tests OF THEIR STATISTICAL SIGNIFICANCE 
por MATHEMATICS AND ENGLISH MAJORS 
BETWEEN ALL INDIVIDUAL EXAMINATIONS 
axp COMPOSITE SCORE IN THE IOWA QUALI- 
ryING EXAMINATIONS 


Part la. Mathe »matics Majors 


Examinations* Diff. o diff. ST deed 
TS) as |. 4.5 2.44 
7! aa 5.1 3.92 
7d) pkweZ 2 ae 
= 13 68 1.91 
OF 1, Sa 1.3 1.51 
Part 1b. English Majors 

“HSC-MA....-------- 11 42 262 
1L.S.R.-M.A. Se 4.5 2.44 
POY Sea 16 42 £381 
CEE: anmnitawnn EE 4.4 2.73 


*The examination with the highest mean 
percent tile score is listed first. 

** For these two groups “t’”? must exceed 
258 to be significant at the 1% level and 1.96 
at the 5% level. 


ference in means.* The variance of each 
major group in all of the individual exam- 
inations and composite score was analyzed. 
By applying the “F” test, the results of which 
are presented in Table III, it was found that 
no significant** differences in means existed 
within any of the groups except for the Math- 
ematics and English Majors. These differ- 
ences and tests of their significance are pre- 
sented in Table IV. The results indicate that 
the Mathematics Majors perform significantly 
higher in the Mathematics Aptitude Exam- 
ination than in the Iowa Silent Reading Test 


_ and English Training Examination. The data 


also indicate that the English Majors score 
significantly higher in the High School Con- 
tent Examination, English Training Exam- 
ination and composite score than in the 
Mathematics Aptitude Examination. Since 
(here are no significant differences within any 
i the other major groups it does not appear 
advisable for a counselor to rely heavily upon 
the pattern of scores in the individual exam- 
nations for differential prediction. 

The variance of all major groups in each 
if the individual examinations was also ana- 
lyzed. The results are reported in Table V 


*One of the underlying assumptions in analysis of vari- 
ance is normality of the population from which statistics 

ch as the mean are derived and hence the use of percentiles 
$ not fully ~~ It is believed, however, that the result- 
ing errors will not be large. 


“Because of the nature of the data the writers have 
hosen the 1% level as their criterion of statistical signifi- 
cance ay evaluating all of the differences in means found in 


os sty 
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TABLE V 


RESULTS OF ANALYSIS OF VARIANCE WiTHIN 
GROUPS AND BETWEEN GROUP MEANS FOR 
EAcH EXAMINATION AND THE COMPOSITE 
SCORE 

Sum of 
d.f. Squares Variance 
. High School Content Examination 
Between groups 8 21390.8 2673.8 
Within groups. 356 240163.7 674.6 


~ 





ee 364 
F Ratio 3.96* 


2. Iowa Silent Reading Test 


Between groups 8  15293.5 1911.6 
Within groups. 351 223859.9 637.7 
an 35! 9 
F ratio 2.99 

3. Mathematics Aptitude Examination 
Between groups 8 32324.4 4040.5 
Within groups. 356 256496.8 720.4 
EE cinta 364 
F ratio 5.60 

4. English Training Examination 
Between groups 8 31042.0 3880.2 
Within groups. 356 209817.2 589.3 
aE 364 
F ratio 6.58 

5. Composite Score 
Between groups 8 23500.4 2937.5 
Within groups. 356 224692.0 631.1 


, | een 364 
F ratio 4.65 


*In this table “F” must equal 2.56 to be 
ce at the 1% level and 1.96 at the 5% 
eve 


and reveal that within every examination 
there are statistically significant differences 
in means for some of the major groups. The 
differences found to be significant or ap- 
proaching significance are presented in Table 
VI. The mean scores of Mathematics Majors 
are significantly higher than those of other 
major groups in the following examinations: 
Mathematics Aptitude, higher than all other 
major groups; High School Content, higher 
than Biological Science, Journalism, Political 
Science, and Fine Arts Majors; English 
Training, higher than Biological Science, 
General Science and Political Science Majors; 
Composite score, higher than all major groups 
except English and Physical Science. 
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English Majors exhibit significantly higher 
mean scores than other groups in the follow- 
ing examinations: English Training, higher 
than all major groups except Physical Science 
and Mathematics; lowa Silent Reading Test, 
higher than Biological Science, General Sci- 
ence, Political Science and Fine Arts Majors; 
High School Content Examination, higher 
than Journalism, Political Science and Fine 
Arts Majors; Composite score, higher than 
Biological Science, Journalism, Political Sci- 
ence, History and Fine Arts Majors. 


TABLE VI 


SUMMARY OF DIFFERENCES IN MEANS AND 
Tests OF THEIR SIGNIFICANCE FOR ALL 
Masor Groups WITHIN FACH INDIVIDUAL 
EXAMINATION AND COMPOSITE SCORE OF THE 
IOWA QUALIFYING EXAMINATION 


Part 1a. High School Content Examination 


Maior Groups* inn. @¢a18. “E°** 
Math.—Biol. Se. 20 7.0 2.86 
Math.—Gen. Sc. 15 7.0 2.14 
Math.—Journ. __ 2 21 5.5 3.82 
Math.—Pol. Se. Scar ae 5.4 3.33 
Math.—Hist. ot EO 5.9 2.54 
Math.-Fine Arts __ _ 28 6.2 4.52 
Phys. Se.—Fine Arts ~ 1.2 2.50 
Eng.—Biol. Se. mabe 15 6.0 2.50 
Eng.—Journ. —__- . 16 4.1 3.90 
Eng.-—Pol. Se. __-- ee 4.0 3.25 
Eng.-Hist. —~ ; 10 4.6 2.17 
Eng.—Fine Arts . 23 5.0 4.60 
Hist.-Fine Arts eter 5.9 2.20 
Part 1b. Iowa Silent Reading Test 
Math.—Pol. Se. __._...... 14 5.8 2.41 
Math.—Fine Arts — 6.0 a iy f 
Eng.—Biol. Se. — aioe AE 6.3 2.70 
Eng.—Gen. Sc. — ee 16 5.6 2.86 
Major Groups* Df. dt. “"** 
OE 18 4.2 4.29 
are 14 5.9 2.37 
Eng.—-Fine Arts ________ 17 4.4 3.86 


Part le. Mathematics Aptitude Examination 


Math.—Phys. Sc. _______ 17 5.4 8.15 
Math.-Biol. Se. ________ 34 5.6 6.07 
Math.—Gen. Se. _______- 22 4.7 4.68 
Men—nne., ........... 27 3.6 7.50 
weetn-JOUrR. ........_. 32 3.8 8.42 
Math.—Pol. Se. _________ 34 3.5 9.71 
Math.-Hist. _._________ 38 5.2 7.31 
Math.-Fine Arts ______ 39 4.7 8.30 
Phys. Se.—Biol. Se. _____ 17 7.6 2.24 
Phys. Se.—Journ. ~_____ 15 6.4 2.34 
Phys. Se.—Pol. Se. ~____ 17 6.3 2.70 
Phys. Sc.—Hist. .....__- 21 7.3 2.88 
Phys. Se.—Fine Arts ____ 22 7.0 3.14 
Gen. Se.—Hist. _._______ 16 6.7 2.39 
Gen. Se.—Fine Arts ____ 17 6.4 2.66 
Eng.—Fine Arts ________ 12 5.7 2.11 





[Vo 


Part 1d. English Training Examinat 


Major Groups* Diff. o diff. 
Math.—Biol. Se. __ ina 7.6 
Math.—Gen. Se. _____- 20 7.6 
meen-ro. Se. ......... 19 6.4 
Eng.—Phys. Sc. ________ 16 6.7 
Eng.—Biol. Se. ______- 27 5. 
Eng.—Journ, __..--_____ 15 9 
Eng.—Gen. Se. _____- 27 f 
Eng.—Pol. Sc. __..-- 26 3.9 
Eng.-Hist. — _ 20 19 
Eng.—Fine Arts ____- 12 1.6 
Fine Arts—Biol. Se. _-__ 15 6.3 
Fine Arts—Gen. Se. __ 15 6.4 
Fine Arts—Pol. Sc. _____ 14 1.9 9 
Journ.—Biol. Se. __.____ 12 ( ? 
Journ.—Pol. Se. __- 1] 1.4 


Part le. Composite Score in Qualifying 


Examination 

Math.-Biol. Se. __ 2 7.4 

Math.—Gen. Se. _____- 17 

Math.—Journ. ___ er 

Math.-Pol. Se. ____ 25 7 

Major Groups* Diff. od 

Math.—-Hist. _____- eaneoe Se 6.8 

Math.—Fine Arts _______ 26 

Eng.-Biol. Sc. _________ 19 6.3 

Eng.Journ. ........... 12 1.2 2 

ES aS i 1 4.2 

Eng.—Hist. _..._....._._. 15 2 

Eng.—Fine Arts ____- ee 

*The major group with the highest mea: 
percentile score is listed first. 

** For all these groups “t” must exceed 2.58 


to be significant at the 1% level and 1.96 # 
the 5% level. 


Physical Science Majors show a. signil 
cantly higher mean in the Mathematics Apt:- 
tude Examination than Political 
History and Fine Arts Majors. General S¢- 
ence Majors exhibit a mean score in Math 
ematics Aptitude which is significantly higher 
than that of the Fine Arts Majors. The mean 
score of Fine Arts Majors is significantly 
higher in English Training that that of the 
Political Science Majors. Several other difier 
ences are quite large but fall slightly below 
the magnitude required for significance at th 
one per cent level. 


Some interesting and helpful observation: 
may be made concerning these significant «i'- 
ferences. The highest ranking groups in con 
posite score, namely, Mathematics and Eng: 
lish Majors, are most frequently significantly 
higher than other groups in the individual 
examinations. The third ranking group, Phys 
ical Science Majors, is also third in frequency 
of significantly high scores in the individual 


science 
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examinations. It is worthy of note that Eng- 


-) Majors are significantly higher in Com- 
; site score and High School Content Exam- 
in than Journalism Majors. Fine Arts 
Majors, while characteristically lower than 
he other groups in most instances, score sig- 
ily higher than Political Science Majors 
» the English Training Examination. In this 
ection it should be pointed out that high 
mic aptitude is probably not as impor- 
+ 9 factor in determining success in Fine 
{rts as in the other academic areas. Tests 
ich as those investigated in this study should 
st certainly be supplemented with others 
nseling students concerning their prob- 
success in art. 


CONCLUSIONS 


On the basis of the results of this study it 
ypears that a knowledge of the distribution 
scores of successful majors in various aca- 
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demic departments should prove helpful in 
counseling students concerning their choice of 
a major subject. Students ranking high in 
the Iowa Qualifying Examination may be 
afforded wide choices of major subjects; 
however, the choice of those students ranking 
low, if they wish to compete with students of 
their own abilities, is more restricted. As this 
study dealt only with students who had suc- 
cessfully completed work in various academic 
departments, no absolute lower limits of per- 
formance in the Iowa Qualifying Examination 
can definitely be established. However, if one 
were willing to assume that the level of per- 
formance represented by students included in 
this study is indicative of the level of ability 
required for success in each of the major 
fields, then the results of this investigation 
should be of some value in counseling stu- 
dents with regard to their academic majors. 
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THE RELATIONSHIP BETWEEN SCORES ON THE Cayp 
INTELLIGENCE SCALE AND SUCCESS IN GRADUATE WORK 
AT COLORADO STATE COLLEGE OF EDUCATION 


LORAINE BRUCE 


Pampa Senio 


r High School 


Pampa, Texas 


1. PROBLEM 


The object of this study was to determine 
the validity of the I.E.R. Intelligence Scale 
CAVD, Levels M, N, O, P and Q for the pur- 
pose of predicting achievement at the master’s 
level in the Colorado State College of Educa- 
tion. The CAVD Scale is made up of four 
series of tasks: completions, arithmetical 
problems, vocabulary and directions. The 
upper five levels are for use in higher under- 
graduate and graduate divisions of the col- 
lege. 


2. SUBJECTS AND ‘TESTS 


All students matriculating on the master’s 
level from June, 1936 to August, 1938 were 
given the I.E.R. Intelligence Scale CAVD, 
Levels M, N, O, P and Q, Form 4, and the 
American Council on Education Cooperative 
English Test. Some of these students took the 
battery of twelve education tests given to 
those who major in education and who have 
elected the four quarter plan for the master’s 
degree. Most of the students on whom the 
A.B. degree was conferred by Colorado State 
College of Education had taken the American 
Council on Education Psychological Exam- 
ination while they were undergraduates. 


3. RESULTS 


Of the students on the master’s level who 
took the CAVD, 440 were in the graduate 
school at least two quarters and took a min- 
imum of twelve quarter-hours each quarter. 
The majority of this graduate work was done 
during the summer quarters. In order to com- 
pute the reliability of teachers’ marks and 
other correlations in which teachers’ marks 
were involved, the point-hour ratio was found. 
The point-hour ratio was computed by divid- 
ing the total number of points by the total 
number of hours. The points were found by 
multiplying the number of each course by the 
point values of the letter grades. The point 
value of “A” is 5; of “B” is 4; of “C”, 3; 
of “D”, 2; and of “F”, 1. The reliability of 


teachers’ marks as found by computing th 
product-moment coefficient of correlation bye. 


tween the point-hour ratios for the first quar. 
ter and the second quarter for this group oi 
440 students is .467 + .025. This is a re. 


atively low reliability coefficient. The 
product-moment coefficient of correlation be. 
tween scaled scores on the CAVD Intelligence 
Scale and the total point-hour ratio of th 
first two quarters of graduate work wa: 
310 + .029. Although this is a significan 
correlation, yet it is low if the CAVD ist 
be used as a predictive measure for teachers 
marks. 

The means of the point-hour ratios com- 
puted from the first quarter of twelve 
more hours were found for two groups; first 
the group making below the 25th percentile 
and second, the group making scores above 
the 75th percentile on the CAVD. The mean 
of the first group (V = 243) was 3.72 and 
the mean of the second group (.V 142 
was 4.25 and the difference between thes 
means was .53. Since the probable error o! 
the difference was .037, there is virtual cer- 
tainty that there is a significant differenc: 
in fact, using the ratio, D/PE,, the chances 
are better than 99.99 in 100 that this is a 
true difference. 

There were 134 students on the masters 
level who took both the CAVD Intelligence 
Scale and the American Council on Education 
Psychological Examination and who ha 
taken at least twelve quarter-hours in the 
graduate school. A product-moment coeli- 
cient of correlation of .752 = .025 was loune 
between scaled scores on the CAVD Inte!» 
gence Scale and sigma scores on the America 
Council Psychological Examination. Perce 
tiles on the American Council Psychologic 
Examination were translated into sigma sco" 
by using a distribution truncated at 
In order to avoid negative o values, the 2°" 
point was put at the lower end of the sc 
which in this case was at —5 a. The corre's 
tion is significant and indicates a relationship 
53 per cent better than pure chance. When 4 
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2 
the percentage of agreement between the two 
cries. Using the same group of students, the 
coefficient of correlation was .232 + .055 be- 
tween scaled scores on the CAVD Scale and 
total point-hour ratio of marks on the grad- 
yate level. This indicates only a slight corre- 
lation. A third coefficient of .304 + .053 was 
found between sigma scores on the American 
Council Psychological Examination and the 
total point-hour ratio of marks on the mas- 
ter's level. This is a slightly better correlation 
than the preceding one. The coefficient of 
522 indicates a correlation 31 per cent better 
than chance and that of .304 a correlation 33 
per cent better than chance. 

The difference between the two preceding 
correlations (.23 and .30) is .o7 and the 
probable error of the difference is .og1. In 
order to find out whether this is a significant 
difference, the following formula? for the 
probable error of the difference between two 
coefficients of correlation was used: 


centage of disagreement and 1— 4 
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In order to complete matriculation, mas- 
ters’ students were required to take the 
American Council Cooperative English Test; 
the parts given were English Usage and Spell- 
ing. The product-moment correlation (NV == 
748) between scaled scores on the CAVD 
Intelligence Scale and scaled scores on the 
English Usage Test was .553 = .017; that 
between scaled scores on the CAVD and 
spelling .502 + .o19. These r’s show sub- 
stantial relationships but they also indicate 
that different abilities were measured. 

Until the summer of 1936, there was only 
one method, the thesis plan, of working for 
a master’s degree in Colorado State College 
of Education. Beginning then, graduate stu- 
dents were given a choice of two programs of 
work leading to the master’s degree. The 
thesis plan was one of the two programs; the 
second method was known as the examination 
plan. One of the three required comprehen- 
sive examination programs covered the gen- 
eral field of education. In the summer of 
1937, the thesis plan was called the A plan 
and the second plan was changed to the B 
or four quarter plan. Those who had started 
the examination plan in 1936 and who wished 
to continue were allowed to do so. 
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This formula took account of the sampling 
correlation between the correlation coeffi- 
cients or, stated differently, took account of 
the effect of using the same group. The cor- 
relation is not significant. 


Before 1936, the American Council Psy- 
chological Examination was the intelligence 
test used for masters’ students. When this 
test was used, the distributions of scores were 
negatively skewed. The distributions of scores 
on the CAVD more nearly approach a normal 
curve. In Colorado State College of Educa- 
tion, the CAVD Scale has proved to be a 
more practical scale for graduate students. 


‘Heilman, J. D. “The k and Methods of Interpretin: 
the Coefficient of Correlation.” J : of Educational Psy. 
chology, 28, 232-236 (March, 1937). 


* Peters, Charles C. and Van Voorhis, Walter R. Statistical 
pipedures and Their Mathematical Bases, p. 155, School of 


Education, Th A 
Pennsylvania, — State College, State College, 





All students who elected the comprehensive 
examination plan and all who elected the four 
quarter plan with education as their major 
were required to take a battery of twelve ed- 
ucation tests and to pass any nine of the 
twelve. This battery of education tests con- 
sists of tests in the following subjects: tech- 
niques of curriculum making, elementary edu- 
cation, history of education, mental tests, 
personnel and guidance, philosophy of educa- 
tion, psychology of learning, research, school 
administration, secondary education, educa- 
tional tests, and statistics. The tests were 
made up by various faculty members of Colo- 
rado State College of Education and were 
revised by the Personnel Department of the 
college. There are two equivalent forms of 
each test. All the items of each test are 
multiple-choice of four choices. The number 
of items in the various tests ranges from forty 
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for research to fifty-seven for personnel and 
guidance. The passing mark on each test was 
determined from the group of ninety-one who 
first took the tests in the summer of 1936. 
The passing point was selected at approx- 
imately the 55th percentile. The percentile 
point for statistics and mental tests was some- 
what higher. 

A comparison was made of scores on the 
CAVD Scale and scores on this battery of 
twelve education tests the first time they were 
taken. The quartile deviation (Q) of each 
education test was found, and the scores were 
given equal weights on the basis of the rel- 
ative sizes of the Q’s; then the average of the 
weighted scores was found, with nine as the 
minimum number of tests used. A product- 
moment coefficient of correlation (N == 210) 
of .530 + .034 was obtained between scaled 
scores on the CAVD Scale and this average 
of weighted scores on the education tests. 
This coefficient indicates a substantial rela- 
tion, 40 per cent better than pure chance. 
A coefficient (.V 63) of —.377 + .072 was 
found between scaled scores on the CAVD 
Scale and the total number of education tests 
which were taken before nine were passed. 
This shows a slight relationship between a 
small CAVD score and a large number of 
education tests taken in order to pass nine. 

A study was made of the relation between 
scores on the CAVD Scale and the choice of 
a program of graduate work leading to the 
master’s degree: (a) the thesis plan and 
(b) the four quarter and examination plans. 
Distributions of scaled scores on the CAVD 
tests were made for these two groups. The 
mean of the first group (NV == 323) was 
406.18 and the mean of the second group 
(N == 431) was 407.07. The difference be- 
tween these means was .89 and the probable 
error of the difference of the means was .49. 
There are 89 chances in 100 that the obtained 
difference is significant; this difference is in 
favor of the group selecting the second plan, 
but it is not a true difference. The standard 
deviation of the group which selected the 
first plan was 9.54 + .25 and of the second 
group 10.15 + .23. The difference between 
these two standard deviations was .61, and 
the probable error of the difference between 
these two uncorrelated o’s was .35. The 
ability as measured by the CAVD Scale is 
somewhat higher for the group electing the 
four quarter plan, but the obtained difference 
is not significant; the range of ability covered 


by this group is slightly wider and again the 
difference is rot significant. 


4. CONCLUSIONS 

The I.E.R. Intelligence Scale CAy| 
Levels M, N, O, P and Q is a discriminatiy 
instrument for determining the range of ab. 
ities of graduate students on the master: 
level. 

The correlation between the CAYD Ss DE\ 
and teachers’ marks on the graduate level 
relatively low. The CAVD Scale is sufficient), 
discriminating, however, to make it virtua)! Du 
certain that those students whose scores ar ber Ol 
above the 75th percentile on the CAVD wi tn 
make higher marks on the average than thos wal 
students whose scores on the CAVD lie below nate 
the 25th percentile. US 

There is a marked correlation between th - 
CAVD Scale and the American Council purp 
Education Psychological Examination. T| _s 
correlation between the American Counci tude 
Psychological Examination and __ teachers 
marks on the master’s level is slightly higher Lloret 
than that between the CAVD Scale and teach- - 
ers’ marks, but neither is very high and the 
difference is not significant. It is more prac- 
tical to use the CAVD Scale for graduate stu- 
dents than the American Council Psycholog- 
ical Examination. When the American Coun- 
cil Psychological Examination was used for 
master’s students, the scores were negativel colleg 
skewed. The scores on the CAVD more nearly Th 
approach a normal distribution. Evident'y ward 
marks given to masters’ students in the sum- racy 
mer are based in the main on factors other Pg 
than native ability as measured by intelli- past 
gence tests. used 

Although they evidently measure different varial 
abilities, substantial relationships were found proce 
between the CAVD Scale and the Coopera- path 
tive English Usage Test, and between the a 
CAVD Scale and the Cooperative Spelling grin 
Test. indep 

There is a marked relationship between the studi 
CAVD Scale and the battery of education studi 
tests. A slight negative correlation was found 
between scores on the CAVD Scale and the — 
number of education tests that were taken a | 
before nine were passed. — 

The scores on the CAVD of students who 
elect the four quarter plan for a master’s de- 
gree are not significantly higher than the 
scores of those who elect the thesis plan. The a 
first group shows a wider range of abilities f the 
than the second group. — 
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PREDICTION OF COLLEGE MARKS 


Curtis T. Lear 


Instructor in 


Psychology, 


La Salle—Peru 


Oglesby Junior College, La Salle, Illinois 


PART I 
DEVELOPMENT OF A REGRESSION 
EQUATION* 


During the past two decades a large num- 

her of investigators have concerned themselves 

ith the problem of the relationship between 

the students’ marks in college and_ their 

irks in high school, and the scores on vari- 

examinations which were administered 

ior to their entrance into college. The main 

e of these investigations has been to 

criteria for accurate prediction of the 

tudent’s achievement in college. The studies 

shown definitely that it is impossible to 

the scholastic achievement of all the 

nts in a given group, and that many 

of success or failure can not be discov- 

til the student has tried to do college 

However, it would be advantageous to 

he student and the college to determine 

s accurately as possible beforehand the stu- 

lent’s prospects for success or failure in 
lege work. 

These studies have shown a tendency to- 
ward basing the predictions of the student’s 
future status in college upon objective rather 
than subjective evidence. In some instances 
the zero-order coefficients of correlation were 
used to measure the relationship between two 
variables, in other instances the more exact 
procedures of partial and multiple correlation, 
path coefficients, and coefficients of determi- 
nation were used to determine the relationship 
between a dependent variable and a group of 
indepe’ ut variables. Some _ investigators 
studied the procedure for one year; others 
studied the procedure for several consecutive 
years, and were able to determine the most 
valid predictors, the best single predictor, 
and the best group of predictors for their 
respective colleges. 


THE INVESTIGATION 


_ The purpose of this study is to develop a 
‘ive-variable regression equation which will 


a4 A field study which was submitted in partial fulfillment 
f the requirement for the degree of Doctor of Philosophy in 
lorado State College of Education, Greeley, Colorado, 1938. 


predict with as little error as possible the 
status of college freshmen at the time of their 
entrance into college. The equation is based 
on data obtained on 97 freshmen students at 
the La Salle-Peru—Oglesby Junior College, 
La Salle, Illinois, 1937-1938. The data in- 
clude the average college mark for the entire 
freshman year as the dependent variable, or 
criterion; and the American Council Psycho- 
logical Examination score, the Iowa English 
Aptitude Examination score, the Iowa High 
School Content Examination score, and the 
average high school mark for the four years 
as the independent variables. 

The zero-order coefficients of correlation, 
their probable errors, and the mean and sigma 
of each of the variables are shown in Table I. 
Leinbaum! found a coefficient of correlation 
of .52 between average college marks and 
average high school marks. Scott? reported a 
coefficient of .64 between average college 
marks and the American Council Psycholog- 


TABLE I 


ZERO-ORDER COEFFICIENTS OF CORRELATION BE- 
TWEEN VARIABLES USED IN THE PREDICTION 
OF THE AVERAGE COLLEGE MARKS OF 97 
FRESHMEN STUDENTS AT THE LA SALLE— 
PERU—OGLESBY JUNIOR COLLEGE, LA SALLE, 
ILLINOIS, 1937-1938 


Variables: X,—the average college marks for 

the entire freshman year; 

X.—the American Council Psycho- 
logical Examination score; 

X:—the Iowa English Aptitude 
Examination score; 

X,—the Iowa High School Content 
Examination score; and 

X.—the average high school mark 
for the four years. 
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Mean ___ 3.39 179.5 
Sigma __ 0.73 47.57 
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ical Examination. Hovde* used percentile 
rank on the American Council Psychological 
Examination instead of raw score, and re- 
ported a coefficient of correlation of .55 be- 
tween the American Council Psychological 
Examination and average high school marks. 

In this study, the regression equation in 
score form is: X, == —.0002X, + .0163X, 
+ .0059X, + .5217X, + .0378, (1). This 
means that the student’s predicted mark 
equals —-.0002 times his score on the Amer- 
ican Council Psychological Examination plus 
.0163 times his score on the Iowa English 
Aptitude Examination plus .oo59 times his 
score on the lowa High School Content Exam- 
ination plus .5217 times his average high 
school mark plus the constant, .0378. 

By using this regression equation, the 
average college marks of the 97 students are 
predicted. There are three instances in which 
there are no differences between the predicted 
marks and the actual marks. These are in- 
stances of perfect prediction. The largest 
difference between the predictec and actual 
marks is 1.12 of a letter mark. The median 
of the differences is .26; the probable error 
of estimate, .30; and the standard error of 
estimate, .44 of a letter mark. All of the pre- 
dictions fall within three standard errors of 
estimate, 97 per cent within two, and 71 per 
cent within one; and 58 per cent within one 
probable error of estimate. 

The standard error of estimate of .44 means 
that two-thirds of the marks should be pre- 
dicted within .44 of a letter mark. For prac- 
tical certainty, the range of possible varia- 
tions must be increased to three sigmas, or 
1.32 of a letter mark. Therefore, if a student 
receives the predicted mark of “C”, the mid- 
point of the “C’’-range, the chances are 74 in 
100 that he will receive a mark of “C”, and 
it is practically certain that he will receive 
a mark of not more than “B”, nor less than 
—. 

The coefficient of correlation between the 
predicted marks and the actual marks is .77, 
a significant relationship. Leuenberger* re- 
ported a relationship of .70. 

The coefficient of multiple correlation is 
computed from the regression coefficients and 
the zero-order coefficients of correlation. The 
coefficient of multiple correlation is .79*, and 
gives predictions 57 per cent better than pre- 


_* This is slightly higher than the .77 reported in the pre- 
vious paragraph on account of the grouping error affecting 
the predicted grades in the latter. 
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dictions by chance. In order to determine the 
total single and joint influences of the fi ‘ 
independent variables upon the criterion. +}, 
coefficients of determination are compyte: 
These coefficients are given in Table I]. The 
total of the measured influences is 6243. ¢, 
62.43 per cent. In a normal distribution. t} 
coefficient of multiple correlation square) 
equals the coefficient of determination. Ther: 
is an approximate equality in this situatiop 
Heilman® reported a coefficient of multiple 
correlation of .7985 and a coefficient of deter. 
mination of 63.73 per cent. 


CONCLUSIONS 
This study, although it contributes nothing 


new in the field of prediction, substantiates 
other studies that have been made, and als 
demonstrates the use of correlations and the 
regression equation in prediction. 

The regression equation developed in this 
study is reasonably accurate, since the pre. 
dicted marks compare so favorably with th: 
actual marks. This is tested by the appro: 
imate equality between the median of the 
differences and the probable error of estimat: 


TABLE II 


COEFFICIENTS OF DETERMINATION BASED ON 
DATA OBTAINED ON 97 FRESHMEN STUDENTS 
AT THE LA SALLE—PERU—OGLESBY JUNIOR 
COLLEGE, LA SALLE, ILLINOIS, 1937-1938 


Variables: X:—the average college mark for 
the entire freshman year; 
X,—the American Council Psycho- 
logical Examination score; 
X;—the Iowa English Aptitude 
Examination score; 
X,—the Iowa High Schoo! Content 
Examination score; and 
X;—the average high school mark 
for the four years. 
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1 2 3 
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The equality is not exact since the distribu- 
‘on is not a normal distribution. Further 
‘ests are found in the significant relationship 
between the predicted and actual marks, in 
the number of predictions falling within one 
probable error and one standard error of esti- 
mate, and in the equality of the means of the 
‘wo distributions. The distribution of pre- 
dicted marks has a smaller range and a 
smaller sigma. 

In this particular group of independent 
variables, the best predictor is the average 
high school mark. This variable makes the 
largest single and joint contributions, as is 
shown by the coefficients of determination. 
The poorest predictor is the American Coun- 
ci] Psychological Examination, which makes 
the smallest single and joint contributions. 
The latter are all negative. The portion of 
the total influences on the criterion not meas- 
ured by the four independent variables is 
17.57 per cent. The addition of other tests, 
especially those which measure qualities and 
capacities not measured by the present tests, 
may reduce the percentage of influences un- 
accounted for in this study. 

Educational guidance presupposes some 
means of predicting college marks in order 
that the student may be directed into the 
type of work that best suits his capacities and 
needs. This regression equation can be used 
to predict the average college mark of 68 per 
cent of the students included in this study 
within the standard error of estimate of .44 
of a letter mark. 

This regression equation is significantly 
valid in predicting the average college marks 
of the students on whom the data were ob- 
tained. The real test of its validity will be 
made through using it in predicting for other 
similar groups of students from the same col- 
lege. In the subsequent investigation, the 
validity of the regression equation is verified 
by using it in predicting the average college 
marks of the freshmen students during the 


year following the development of the regres- 
sion equation. 


PART II 


THE VALIDITY OF THE REGRESSION 
EQUATION* 


The typical investigator chooses a group of 
variables and obtains their measures. He 


Pe. field study which was submitted in partial fulfillment 
{the requirement for the degree of Doctor of Philosophy in 
Colorado State College of Education, Greeley, Colorado, 1939. 
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uses these measures in an appropriate way to 
form a regression equation with which he 
predicts the probable status of all or a rep- 
resentative group of students on whom the 
data were obtained. If the regression equa- 
tion predicts well for the group, he recom- 
mends its use for further predictions in the 
same college, and considers his task com- 
pleted. 

The test of the regression equation has not 
been made at this stage. The investigator 
should make further predictions by obtaining 
like measures on a second similar group of 
students, and by using these measures with 
their respective weights from the first equa- 
tion. The coefficient of correlation between 
the predicted marks and the marks actually 
received by the students of the second group 
measures the validity of the regression equa- 
tion as a predictive instrument for the stu- 
dents at that particular college. 


THE INVESTIGATION 


This study is concerned with obtaining a 
measure of the validity of each of two five- 
variable regression equations used in predict- 
ing the probable average college marks of 
freshmen students during the year following 
the development of the regression equation, 
and with measuring the effect on the criterion 
of adding another variable to the variables 
upon which one of the regression equations is 
based. 

The subjects of the study, chosen by 
chance selection, are two groups of 100 stu- 
dents each. One group attended the La Salle— 
Peru-Oglesby Junior College, La Salle, Illi- 
nois, during the year 1938-1939; and the 
other attended the Colorado State Teachers 
College, Greeley, Colorado, during the year 
1932-1933. 

One regression equation was developed 
from data on ninety-seven freshmen students 
at the La Salle-Peru—Oglesby Junior College 
in 1937-1938. The equation (Part I) in score 
form is: 


X, = —.0002X, + .0163X, + .0o59X, + 
.5217X, + .0378, (1) in which the 
variables are: 

X,—the average college mark for the 
freshman year; 

X,—the American Council Psychological 
Examination score; 

X, — the Iowa English Aptitude Examina- 
tion score; 
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X,—the Iowa High School Content Ex- 
amination score; and 


X, —the average high school mark for the 
four years. 
X,—the Personal Data Scale was the 


added variable. 


The other equation was developed by 
Leuenberger* from data on 283 freshmen stu- 
dents at the Colorado State Teachers College 
in 1931~—1932. The equation in score form is: 

Xo 0428X, + .2877X, 33560X, + 

.6429X, + 99.0944, (2) in which the 
variables are: 


§ the average college mark for the 
freshman year; 

X,—the American Council Psychological 
Examination score; 

X, —the English Test score; 

4 the Elementary Test score; and 

X, —the percentile rank of the student in 


his senior class. 


Like measures on a second group of roo 
freshmen students from each college were 
obtained and used with the weights of the 
regression equation already developed in 
order to make the predictions. The differences 
between the predicted and actual marks were 
determined, and comparisons were made in 
terms of the differences and in terms of the 
per cents of the differences falling within and 
without the probable and standard errors of 
estimate. 

For each group of students, the zero-order 
coefficients of correlation between the pairs 
of variables were computed from the new 
data. The zero-order coefficients were used in 
setting up a set of simultaneous equations, 
the solutions of which gave the regression co- 
efficients. The regression coefficients were 
used in determining the coefficient of multiple 
correlation and the coefficients of determina- 
tion. The shrinkage of the coefficient of 
multiple correlation was also computed. 


SUMMARY OF FINDINGS 


For the La Salle~-Peru—Oglesby Junior Col- 
lege students, the differences between the pre- 
dicted and actual marks ranged from .or to 
1.25 points, and the median of the differences 
was .33 of a point. Forty-nine per cent of the 
differences fell within one probable error of 
estimate, and 66 per cent fell within one 
standard error of estimate. The coefficient of 
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correlation between the predicted and ac 
marks was .78. In the previous study \ ¥ 
similar group of students, the range was a 
points, anc the median of the differences ws: 
.26 of a point. The per cents of the difc. 
ences falling within one probable and pp. 
standard error of estimate was 58 per cen 
and 71 per cent respectively. The coefficien: 
of correlation between the predicted 
actual marks was .77. There were variations: 
between the respective measures of the ty 
groups: The range and the median of ; 
differences; and the per cents falling 
one probable error and one standard error 
estimate; but the variations were not dispy 
portionately large. The regression equation 
developed from the data on the first grou; 
students maintained its validity in predicting 
the average marks of the second group. 

The coefficient of multiple corr 
using the six variables, was .81: using ¢} 
first five variables, it was .79. The presence 
of the added variable increased the coefficient 
of multiple correlation. For the pre 
year’s group, that coefficient was .7 
was a small estimated shrinkage of the 
efficient of multiple correlation, but n 
shrinkage. 

The total influences on the criterion as 
measured by the five independent variables 
was 67.26 per cent; as measured by the four 
independent variables it was 62.43 per cent. 
Singly and jointly, the Personal Data Scale 
measured 4.83 per cent of the total influences 
It measured singly .54 per cent of the total 
influences on the criterion, which was more 
than was measured by either the American 
Council Psychological Examination, or the 
Iowa English Aptitude Examination. 


For the Colorado State Teachers College 
students, the differences between the pre- 
dicted and actual marks ranged from .oo t0 
1.38 points, and the median of the differences 
was .31 of a point. Forty-seven per cent 0! 
the differences fell within one probable error 
of estimate, and 65 per cent fell within one 
standard error of estimate. The coefficient 0! 
correlation between the predicted and actual 
marks was .70. In the original study o/ 4 
similar group, the range was .9o of a puitlt, 
and the median of the differences was .2 0! 
a point. Fifty-one per cent of the differences 
fell within one probable error of estimate, and 
66 per cent fell within one standard error of 
estimate. The coefficient of correlation be 
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PREDICTION OF 


e predicted and actual marks was .7o. 
aa there were increases and de- 
;, the range and the median of the 
i rences, and in the per cents falling within 
oe proba ible error and one standard error of 
_the differences were not significant. 
The r sche equation was as valid in pre- 
ting for the second similar group of stu- 
ents as it was for the group for which it 
is devi loped. 
rhe coefficient of mul itiple correlation was 
-§ for the preceding year’s group it was .73. 
There was a small estimated shrinkage and 
ve actual shrinkage of the coefficient of 
tinle correlation. The total influences on 
criterion as measured by the independent 
riables was 63.15 per cent, and increase of 
per cent over the total for the previous 


W « 


CONCLUSIONS 
this study reports a measure of the valid- 
ty of each of two regression equations used 
n predicting the average college marks of a 
ind similar group of students. Each equa- 
was significantly valid for its own second 


rhe best predictor in this combination of 


variables for the La Salle-Peru—Oglesby 
unior College students was the average high 

| mark; for the Colorado State Teach- 
College students, the Elementary Test. 
he American Council Psychological Exam- 
ination was the poorest predictor for both 
groups of students. 

lhe Personal Data Scale measured some of 
the total influences on the criterion, and 
measured slightly more than either the Amer- 
can Council Psychological Examination or 
he Iowa English Aptitude Examination. 


COLLEGE MARK 


These regression equations should be an 
aid to those who advise the freshmen stu- 
dents who enter these two colleges, since they 
predict the average college mark of approx- 
imately 68 per cent of the students within .44 
and .40 of a letter mark. 
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EVALUATION OR GUIDANCE? 


The Report of the Eighth Annual National College Sophomore 
Testing Program April 17 to May 5, 1939 


Epwarp E. CURETON 
Alabama Polytechnic Institute* 


I. INTRODUCTION 

American education has become evaluation- 
conscious. Objective tests and other instru- 
ments that are not so objective have been 
used and misused to evaluate individuals, in- 
structors, departments, colleges, and even the 
educational systems of entire states. Some of 
this evaluation is significant and valuable. 
Much of it is harmless and also useless. 
Certain progressive philosophers, observing 
that the useless and pernicious cases outnum- 
ber the significant and valuable ones, have 
condemned the whole evaluation movement, 
and the testing and guidance movements along 
with it for good measure. 

Guidance and evaluation are both examples 
of judgment. Evaluation looks backward and 
judges the past, while guidance looks forward 
and judges the future. Since the future, 
fundamentally, is always more important 
than the past, it follows that guidance is in- 
trinsically a more important type of activity 
than evaluation. Both of these activities 
employ measurement as a tool, and both of 
them employ other tools also. One still hears 
occasionally the argument that since compar- 
able objective tests do not constitute a com- 
plete kit for either purpose, they should be 
abandoned. Fortunately, it is hardly neces- 
sary any longer to reply to this argument. 

Evaluation, contrary to popular belief, 
makes much more exacting demands upon the 
science of measurement than does guidance. 
If a test is to be used in the appraisal of the 
effects of instruction upon an individual, a 
class, or some larger group, it must possess 
demonstrable validity for its purpose. This 
means that it must measure with some fidel- 
ity the really important outcomes desired 
from the instruction. These outcomes may be 
in the nature of skills, items of knowledge, 
habits, attitudes, or even character and per- 
sonality traits. Existing tests are much more 
successful in measuring the first three of these 

* Formerly Research Advisor, Cooperative Test Service. 


outcomes than the last two. Philosophers 
agree unanimously, on the other hand, tha 
the last two are more significant than the 
others. If the work of students and teachers 
is evaluated primarily on the basis of objec. 
tive tests, therefore, there is an inevitable 
temptation to neglect the more fundamental 
objectives in favor of the more measurable 
ones. 


The obvious way out of this dilemma is t 
appraise all of the more important desired 
outcomes, using the most accurate methods 
available in each case. In arriving at general 
judgments, it is then necessary to weight each 
appraisal by its intrinsic importance pri- 
marily, and only secondarily by the reliability 
of the measurement or estimate on which it 
is based. The fact that it is difficult to do 
this has occasionally been used as an argu- 
ment to support proposals that objective tests 
be abandoned as instruments of evaluation 
If the difficulty is real, and apparently it : 
real in many cases, it would seem much more 
logical to be cautious about the whole evalu- 
ation process. Judgments about the real 
growth of students toward worthy ends, 
about the effects of specific programs of in- 
struction upon such growth, about the relative 
abilities of different instructors to stimulate 
the process, and about the general excellence 
of a school or school system as an environ- 
ment for nurturing it, should be offered only 
with the greatest caution. Evaluation is 4 
very difficult art, and the tools at present 
available are not only crude on the whole, 
but of highly uneven crudity as well. 


Guidance, on the other hand, imposes 
appreciably simpler requirements upon the 
technique of measurement than does evalua- 
tion. To possess value as an instrument 0! 
guidance, a test needs only to measure some 
function that is important, and to give a mort 
accurate appraisal than is provided by other 
measures of this or of related functions. Tests 
can easily be dangerous as instruments © 
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EVALUATION OR GUIDANCE? 3 


tion, but as instruments of guidance, 
r oy are almost always useful unless grossly 
preted. Suppose that a group of stu- 
‘ents are measured by a battery such as the 
“poperative General Culture Test, contain- 

» among others survey tests in the social 
‘dies and in the natural sciences. Suppose, 
-her, that these two tests measure only 
-nowledge of facts and the more or less 
vious interpretations of these facts, but 
that they are both provided with accurate 

| comparable norms. The social science 
itments of the college might be engaged 
Be y in efforts to develop in their stu- 
lents the attitudes which are the logical con- 
mitants of a democratic philosophy, and 

.e natural science departments might at the 
ame time be bending their energies toward 
giving their students a general appreciation 
i the effects of science upon modern indus- 

ivilization and of the necessity for the 

sion of scientific modes of thought into 

csitaeitian of social questions. If, now, 

proposed to “evaluate” the two de- 

its by comparing the average score in 

with the corresponding norm, great 

um might easily be done. If the “evalua- 

t ns” were made the basis for any effective 

idministrative action, both groups might well 

induced thereby to abandon their progres- 

sive policies and go back to the compart- 

entalized teaching of facts, principles, and 

ious interpretations within their respective 
reas. 

Now consider the individual student. He 
nay be shown ais scores on these two tests 
1 terms of some comparable units, and it 
may be explained to him that the tests 
measure only factual and elementary inter- 
nretative knowledge in their respective areas. 
‘his will not change any of the desirable 
ttitudes and appreciations that may have 
een built up in him as a result of his work 

the various departments. The added 
snowledge concerning his background in the 
‘wo general areas will be a clear gain to him, 
making his future educational decisions and 
possibly even his basic vocational choices 
‘hat much more intelligent. 


The reports from participating colleges 
indicate that up to the present time the re- 
sults of the national sophomore testing pro- 
grams have been used primarily for purposes 
of evaluation. Many colleges have been con- 
‘ent with this, and have continued to partici- 
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pate in the programs year after year. Many 
others, after participating for a year or two, 
have found as much harm as good resulting, 
or have discovered that after giving the tests 
they still possess too inadequate information 
for really effective evaluation, and have 
therefore withdrawn from the programs. 
Some few, possessing adequate personnel and 
guidance staffs themselves, have used the 
tests regularly for guidance purposes, sup- 
plying the necessary interpretative materials 
themselves. 

A definite effort has been made by the 
Cooperative Test Service this year to incresse 
and emphasize the guidance values of ite 
tests used in this program. This effort has re- 
sulted in three innovations. First, the Co- 
operative Contemporary Affairs Test was 
modified to yield six part-scores, on the 
assumption that the pattern of these scores, 
after reduction to comparable units, would 
yield an estimate of the corresponding pat- 
tern of functioning interests of the student. 
A knowledge of these functioning interests, it 
was felt, should prove a valuable supplement 
to the corresponding measures of academic 
achievement in enabling the student to ap- 
praise his total background. Second, prelim- 
inary norms were derived at the earliest pos- 
sible date, and the General Culture and Con- 
temporary Affairs Tests were provided with 
scores approximately equivalent to the Scaled 
Scores of the Cooperative English and Liter- 
ary Comprehension Tests. Third, special pro- 
file charts were drawn up, incorporating these 
Scaled Scores and Scaled Score Equivalents, 
and an effort was made to supply each co- 
operating college with copies of these charts 
before the end of the spring semester. Each 
college was sent without charge as many 
copies of the chart as the number of students 
tested. It was hoped that these charts would 
be filled out and interpreted by the students 
themselves. 


In the second section of this article, we 
present the usual normative and statistical 
data required for purposes of evaluation, to- 
gether with some evaluations of the tests. In 
the third section, an effort is made to discuss 
in greater detail the techniques of using com- 
prehensive comparable tests for guidance 
purposes, with particular reference to the 
needs of colleges which do not possess special 
personnel and guidance officers. 
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Il. EVALUATION 


A total of 28,461 students in 159 colleges 
and universities were measured with one or 
more tests in connection with the 1939 
National Sophomore Testing Program. Of 
these, 118 institutions returned their results 
in time to be used in the tabulations upon 
which this report is based. Since the partici- 
pating colleges used various combinations of 
tests, the tables following are all based on 
different numbers of cases. It is believed, 
however, that the results are approximately 
comparable except for the greater accuracy 
of those based on larger numbers of cases. 


Basic NORMATIVE DATA 


The tests used in the 1939 Program, as in 
previous years, were divided into two sets. 
Those in the first, which were considered the 
most important, were recommended to all 
participants as a minimum program. They 
consisted of the English, Literary Compre- 
hension, General Culture, and Contemporary 
Affairs Tests. A variety of tests was recom- 
mended in the second set, as supplementary 
examinations to be used as local conditions 
seemed to warrant. This report is concerned 
only with the tests of the minimum recom- 
mended program, since no particular one of 
the other recommended tests was used in a 
sufficient number of colleges to warrant the 
preparation of special norms, or the under- 
taking of interpretative studies. 


Table I presents the final percentile norms 
for the tests of the minimum recommended 
program. These norms are based on the re- 
sults returned from coileges in which the Gen- 
eral Culture and Contemporary Affairs Tests 
were administered by the usual process of 
having the students mark their answers in the 
test booklets. Since the English and Literary 
Comprehension Tests are provided with 
Scaled Scores, the tables on the keys and an- 
swer sheets for converting raw scores into 
Scaled Scores provide for any differences in 
the modes of administration of these two 
examinations. 


The growing use of the International Test 
Scoring Machine is reflected in the increasing 
number of colleges using separate answer 
sheets with the General Culture and Contem- 
porary Affairs Tests. While a few colleges 
used separate answer sheets and scored these 
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by hand, it is highly probable that jp , 
of the institutions where separate pes 
sheets were used with these tests, the sev, ws 
was done by machine. In 35 different , 
leges, a total of 11,789 students used Sri 
sheets with either the General Culture Tox 
or the Contemporary Affairs ‘Test 
Twenty-one of these institutions returned re. 
sults in time for tabulation. Table |! 

is based on these returns, gives the bas} 
centiles for the two tests. It has been esta). 
lished in a number of cases that the yse . 

separate answer sheets increases the difficy): 
of any test with which they are employed, s 
that separate norms are necessary. The dif. 
ferences are usually not very large, and they | 
vary appreciably from one test to another : 


The fact that the scores at most percentiles 


in Table II are higher than the « ond: 
ing scores in Table I indicates, therefore, that 
the 21 colleges using separate answer sheets 
which returned results are appreciably above 
the general averages shown in Table | 
Equatings were made by two different 
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ods to determine the additional difficulty 8 
the General Culture and Contemporary 2 fF 
Affairs Tests when used with separate answer Ee 
sheets. The results of this equating are FE» 
presented in Table III. a2 

In a number of institutions, the Coopera- - 

tive General Culture Test, besides being used v 





in the sophomore class, is used in lower and 
higher classes as well, and rough measures o! 
educational growth are obtained by noting a 
student’s gains in score from year to year. 
In order to interpret these measures, it is 
necessary to know what score on one form 0! 
the test corresponds to a given score on ab- 
other form. Table IV gives the scores on 
Form P, used this year, which are equivalent 
to given scores on Form O, used last year. 


FINAL 


In a few institutions, the tests of the min- 
imum recommended program were given t 
all four classes. Even in these institutions, 
however, not all of the tests in the program 
were used. Table V gives condensed percen- 
tiles for all four classes in these colleges. 
Since the number of cases is rather small, 
these results will not be extremely accurate. 
A rough idea of the representativeness of the 
tables may be obtained by comparing per 
centiles for the sophomore class with the cot 
responding percentiles in Table !. 
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INDIVIDUAL AND INSTITUTIONAL 
DIFFERENCES 


The tremendous variability in the back- 
grounds, abilities, and interests of college 
students is by now well known. The fact that 
there are very large differences between in- 
stitutions, as well as between individuals 
within the same institution, is often not ap- 
preciated so clearly. Chart I shows the vari- 
ability of all individuals compared with the 
variability among college averages for the 
total scores on the English, General Culture, 


and Contemporary Affairs Tests. Op 
whole, the spread of the college averages i 
only a little less than half as great ’ 


as that 


hala) 


General Culture is above the ninetieth por. 
centile, and the lowest college in ( pislienee, 
rary Affairs is below the third percentile te 
all individual students. In spite of this as 

variability among college averages, howe ‘ 
it should be noted that within any particy|s; 
college the variation among individual sty 


Udl 


dents may still be expected to be more { 


CHART I 


COMPARISONS OF THE VARIABILITY OF INDIVIDUALS AND OF COLLEGE AVERAGES 


Explanation: The bars in each section of the graph show the variation among indi 


viduals 


and among college averages, respectively. The wide portion of each bar represents the range 
of scores of the middle half of the students, or the middle half of the college averages, on th¢ 


A Le 


designated test. The narrow parts extend to the 16th and 84th percentiles, and the lines at the 
ends extend down to the tenth and up to the 90th percentiles. In the case of the college aver- 


ages, the cross below each bar represents the lowest college average, and the one above 





resents the highest college average. The short cross line near the middle of each bar indicates 


the median college average. 


While this chart is based entirely on percentiles, the scale has been altered to correspond 
roughly to a sigma scale, so that vertical distances are approximately comparable. ' 
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ner cent as great as the variability of 
students in all institutions taken together. 
The fact that large institutional differences 
exist does not greatly lessen the problem of 
ndividual differences within any given col- 
lege or university. 
“The existence of such large differences 
between colleges has a number of interesting 


alah 


since they serve students of such different 
ackgrounds and accomplishments. This is as 
i, should be. Variation among institutions is 
essential if a wide variety of college students 
are to be served adequately. But the old fic- 
tion that a count of credit-hours or a tran- 
script of grades furnishes a comparable rec- 
rd of achievement or ability must be 
:bandoned. One of the most important prac- 
tical uses for comparable comprehensive test 
scores, in fact, is in connection with transfer 
students. A grade-transcript indicates a stu- 


ns must possess different academic aims, 


dent’s accomplishments in terms of the aca- 
demic standard of the institution he has been 
attending. This should be supplemented by a 
test-transcript, showing his accomplishments, 
in the areas measured by the tests, in terms 
of the academic standard of the institution 
he proposes to enter. We need more variation 
among the academic standards of colleges, 
not less. But this variation should be con- 
trolled, deliberate, and open. The ostrich- 
attitude that all “accredited” colleges have, 
or even should have, roughly similar academic 
standards ought to be abandoned once and 
for all. 


COMPARISONS OF SPECIAL GROUPS 


In order to make the following compari- 
sons, a restricted group was selected from 
among the students whose norms are given 
in Table I. This group was composed of 
4588 individuals; 1878 men and 2710 
women. It consisted of all students who had 


CuHartT II 
ACHIEVEMENT OF MEN AND WOMEN STUDENTS 


Explanation: This chart shows the means of 1878 men and 2710 women students (soph- 
ores) on the indicated tests, graphed in terms of national percentiles as in Chart I. 
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taken the English, General Culture, and Con- 
temporary Affairs Tests, and who had filled 
out all of the information requested on the 
cover-pages of the last two. In order to ob- 
tain strict comparability in the results, it was 
limited to individuals who had not used sepa- 
rate answer sheets with either the General 
Culture or the Contemporary Affairs Test. 
The various comparisons which follow have 
been made by constructing profiles of the 
average scores on the subtests of the English, 
General Culture, and Contemporary Affairs 
examinations for each set of groups to be 
compared. 


Men and Women.—This first and _ basic 
comparison is shown on Chart II. On the 
English examination, the women are appre- 
ciably superior in usage and spelling and not 
appreciably inferior in vocabulary. On the 
General Culture Test, the men are superior 
in knowledge of social studies, science, and 
mathematics, and the women in knowledge of 
literature and fine arts. A similar situation 
is found for the Contemporary Affairs Test, 
where the men are superior in knowledge of 
political events, social and economic events, 
science and medicine, and amusements; while 
the women are superior in knowledge of re- 
cent happenings in the fields of literature and 
fine arts. These findings support the general 
observation that women are superior to men 
in the linguistic and artistic fields, while men 
are superior to women in the technical fields 
and in what might be termed general infor- 
mation. In the case of the fine arts test of 
the Contemporary Affairs battery, it will be 
noted that both the men’s and women’s aver- 
ages are above the soth percentile. This 
means that the 4588 individuals in the re- 
stricted group are not exactly representative 
of the entire 5701 used in compiling the 
norms. For all of the remaining comparisons 
in this set, the charts were drawn separately 
for men and for women. 


Major Fields of Study.—On the cover page 
of the General Culture Test, there appears 
the question, “What is your major field of 
study?” Ten possible answers to this question 
are listed, and a space is provided for writing 
in an eleventh. The item “miscellaneous” on 
the chart refers to items listed in this last 
category. Charts III and IV give the profiles 
of average scores of students majoring in these 
various fields. The most noteworthy feature 
of Chart III is the extreme variability of 


these groups on the literature and mathe. 
matics tests in the General Culture battery 
The highest scores in literature, strangely 
enough, are found among the group majoring 
in classical and modera languages, rather 
than among those majoring in English. Thjs 
may be due to the fact that the test measyre: 
knowledge of world literature, rather than o' 
English and American literature specifically 
The highest peak on the chart, naturally 
enough, is the point representing the superi- 
ority of mathematics majors on the mathe- 
matics test. Chart IV shows much less yari- 
ability on the whole than does Chart II] 


This may indicate that women base their 
choices of major fields upon less careful self- 
appraisals than do men, or it may merel; 


show that women exhibit a more uniform 
scholastic motivation than do men. The out- 
standing feature of Chart IV is again th 
mathematical superiority of mathematics 
majors. The superiority of the music majors 
on the fine arts sections of both the General 
Culture and Contemporary Affairs Tests is 
also noteworthy. The most consistently high 
scorers are the men classical students, and th: 
most consistently low scorers are the women 
students of business administration. 


Professional Goal.—A second question on 
the cover page of the General Culture Test 
is “What is your professional goal?” Nine 
professional goals and one additional cate- 
gory, “Uncertain”, are listed as possible an- 
swers. Charts V and VI give the profiles o! 
men and women students checking these 
various professional goals. Men students o/ 
engineering show the expected superiority in 
science and mathematics and a concommitant 
inferiority in English and social studies. 
Since there are only two women students 0! 
engineering in the entire group, no profile has 
been plotted for this subgroup. The most 
noteworthy fact brought out by these charts 
would seem to be the consistently high scores 
of students of journalism. In the case of men 
students, this superiority is reduced sull- 
ciently on the fine arts, science, and mathe- 
matics sections of the General Culture Test 
to put their total scores below those of the 
college teaching group on that test. The 
women students of journalism, on the other 
hand, are almost at the top of the fine arts 
section, as well as of the social studies and lit- 
erature sections, and they are above the aver- 
age even on the science and mathematics 
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~ections. Their total scores on the General 
Culture Test, therefore, as well as on the Eng- 
ish and Contemporary Affairs tests, exceed 
‘hose of any other group. It is of particular 
nterest to note that the “uncertain” groups 
re near the medians for both men and women. 
This would seem to call in question the state- 
~ent, sometimes made, that students who 
have a definite professional goal tend to do 
-onsistently better work than those who do not. 


Cultural Influences——A third question on 
the cover page of the General Culture Test 
is, “Which of the following do you think has 
‘ontributed most to your general cultural 
evelopment while in college?” Eight possible 
answers are listed, and a space is provided 
for writing in additional ones. Charts VII 
and VIII give the profiles of students having 
various opinions on this subject. On the 
men’s chart, one is struck immediately by 
the consistent superiority of those who believe 
that independent reading has been the great- 
est contributor. At the bottom we find, also 
with considerable consistency, those who be- 
ieve that the principal contributor was labo- 
ratory work or tutorial or small group confer- 
ences. On the women’s chart, on the contrary, 
the group who believe tutorial or small group 
conferences to have been the greatest con- 
tributor to their cultural development have 
on the whole the highest scores of any. Their 
scores are hardly superior, however, to those 
of the group who believe independent reading 
to have been the greatest contributor. At the 
bottom of the women’s chart we again find 
laboratory work. Probably the reason that a 
belief in the cultural value of tutorial or 
small-group conferences characterizes good 
women students is that the group contains 
certain women’s colleges with a highly selec- 


tive admissions policy which use the tutorial 
system. 


Preferred Periodical—On the cover page 
of the Contemporary Affairs Test there ap- 
pears the item, “Encircle the number corre- 
sponding to the periodical which you like 
best to read.” Ten periodicals are listed. 
Though no additional space was provided, a 
considerable number of students wrote in 
some other name, and these were all classified 
as “miscellaneous” in the preparation of 
Charts IX and X. No very clear trends seem 
apparent in these charts. Men readers of the 
Scientific American are superior in science 
and mathematics, as might be expected, and 
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men readers of the Saturday Review of Liter- 
ature seem to Le somewhat superior in gen- 
eral to the readers of other magazines. This 
finding is even more pronounced in the case 
of women readers, but oddly enough, the 
readers of Poetry appear fairly close to the 
bottom of the group of women. 

Preferred Activities—The cover page of 
the Contemporary Affairs Test contains also 
the request, “Encircle the number correspond- 
ing to the type of activity in which you are 
most interested.” Five groups of activities are 
listed, corresponding to the five subtests of 
the Contemporary Affairs Test other than 
amusements. Charts XI and XII show the 
profiles of those who prefer these various 
types of activities, and an added category 
“miscellaneous” has been added to cover 
other preferred activities written in. The one 
obvious and noteworthy feature of these 
charts is the definite superiority of those ex- 
hibiting a preference for the activity, “Writ- 
ing for student publications: belonging to a 
literary club.” 


Fields of Interest——A third item on the 
cover page of the Contemporary Affairs Test 
is the request, “Encircle the number corre- 
sponding to the field in which you are most 
interested.” Five fields are listed, correspond- 
ing almost precisely to the five subtests other 
than amusements on the Contemporary 
Affairs examination, and a sixth category was 
added on Charts XIII and XIV to include 
other fields of interest written in. The con- 
sistently high scores of those professing in- 
terest in literature is again a noteworthy fea- 
ture. A comprehensive study of the validity 
of the Contemporary Affairs examination as 
a measure of functioning interests is now in 
progress. 

Summary.—Throughout these various com- 
parisons, the high scoring students have been 
those who seem to possess linguistic interests. 
Students who major in the classics, who in- 
tend to become journalists, who believe that 
independent reading has contributed most to 
their cultural development while in college, 
who prefer to read the Saturday Review of 
Literature, who like to write for student pub- 
lications and belong to literary clubs, and who 
profess a general interest in literature seem to 
make high scores oftener than do the others. 
The exceptions are usually their scores in sci- 
ence and mathematics. All this would seem to 
argue for the point that linguistic ability is 
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rather general, affecting knowledge of the 
social studies, the fine arts, and contempo- 
rary happenings as well as of English. It 
would also seem to suggest that the test bat- 
tery is heavily loaded with measures of this 
trait. 


COMPARISONS OF GOOD AND PooR STUDENTS 


For the following comparisons, two new 
groups were obtained, consisting approxi- 
mately of the highest and lowest tenth of the 
students in each college concerned in the pre- 
vious comparisons. This procedure was used, 
instead of the alternative one of picking the 
highest and lowest ten per cent of individuals 
from the total group, in order that differences 
between one institution and another should 
not enter into the comparisons. The group 
hereafter designated as the “upper ten per 
cent” consisted of the 455 individuals, from 
the total group of 4588, having the highest 
total scores on the Cooperative General Cul- 





|Vol 5 \ 


ture Test in their respective colleges. 7», 
group to be called the “lower ten per cent 
consisted of the 455 constituting the ten »»- 
cent, approximately, in each college. hav 
the lowest total scores on this examinat),- 
For these two groups various comparis 
have been made in terms of the data recor; 
on the cover pages of the General Culture 4: 
Contemporary Affairs examinations, and +) 
Contemporary Affairs Test profiles of the : 
groups have becn plotted also. 
Contributions to Cultural Developmen: 
The numbers in the upper and lower ten pe: 
cent checking the various contributors : 
cultural development are shown in Chart X\ 


The only iarge difference on this chart wou: 
seem to be the one concerned with independ. 
ent reading. Good students appear to rate 


independent reading much higher than , 


poor students. The poor students rate clas: 


discussion a little higher than do the ¢ 
students, and both groups rate indepen 


Cuart XV 


APPRAISALS OF CONTRIBUTORS TO CULTURE MADE BY GOOD AND Poor STUDENTS 


Explanation: This chart is based on the replies of students who ranked in either th 
highest or the lowest tenth of the distribution of students in each college on the Genera 


Culture Test. The numbers at the left indicate the number of times each factor was checked 


by students who felt that this factor had contributed most to their cultural development. 


Contributors to Cultural Development 


Ind. Class Course Lab. 
Read'g Disc. Lect. Work 
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Extra- Informal Faculty Other 
Conf, curric. Disc. Disc. 
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CHART XVI 


PERIODICALS PREFERRED BY GOOD AND POOR STUDENTS 


Explanation: This chart is based on the replies of students who ranked in either the 
rest or the lowest tenth of the distribution of students in each college on the General 
‘iture Test. The numbers at the left indicate the number of times each periodical was 


ecked by students preferring it. 
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reading, class discussion, and course lectures 
more highly than they rate laboratory work, 
tutorial or small group conferences, extra- 
curricular activities, informal discussions with 
students, or informal discussions with faculty. 

Prejerred Periodical—The numbers in the 
two groups who prefer to read various peri- 
dicals are shown in Chart XVI. No large 
(ifferences appear here, though the preference 
{ good students for the Scientific American 
may be slightly significant. 

Preferred Activity.—Chart XVII shows the 
numbers in the two groups preferring differ- 
ent types of activities. Appreciable differences 
ccur in these groups. The good students 
appear to prefer literary and scientific activ- 
ities appreciably more often than do the poor 
students; and the poor students appear to 
preler social and economic activities notice- 
ably more often than do the good students. 





Field of Interest —Chart XVIII shows the 
numbers in the two groups having different 
stated major fields of interest. Here again 
noticeable differences appear. Good students 
seem most often to be interested in literature 
or in science and medicine, while poor stu- 
dents are interested appreciably oftener in 
social and economic events. 

Contemporary Affairs Profile-—Chart XIX 
shows the profile of average scores on the 
Contemporary Affairs Test of students in the 
upper and lower ten per cent on the General 
Culture Test. From this chart, it is obvious 
that good students know more about Con- 
temporary Affairs than do poor students, and 
that the difference is relatively uniform from 
one field to another. The good students show 
an expected slight relative dip on amuse- 
ments, though they still surpass the poor stu- 
dents on this item by a substantial amount. 
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CHART XVII 
ACTIVITIES PREFERRED BY GOOD AND Poor STUDENTS 


Explanation: This chart is based on the replies of students who ranked in either + 
highest or the lowest tenth of the distribution of students in each college on the Gene... 
Culture Test. The numbers at the left indicate the number of times each activity was »., nigne: 
ferred by these students. . a aoe Cuitu 
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Ill. GUIDANCE is an integral aspect of the American ce! sional 
a Te sla a ‘ cratic philosophy has been successful in re- pensi 
aa guidance apceeagae has been growing straining guidance officers and teachers {r0’ of suc 
Steadily in recent years. Its development has placing unwarranted confidence in test scores not re 
not been as rapid as has that of the evalua- jn the actual counseling situation. deal 
tion movement, but on the other hand it has The major obstacle in the way of the efiec- regulz 
been somewhat more consistent. Opposition tive development of the guidance movement some 
has been passive rather than active; a matter has been, if anything, a too great skepticiss make 
of inertia rather than of any definite objec- regarding the value of objective tests for this basis 
tions. It has been easy to misuse tests in eval- purpose. This skepticism has been effective!) Th 
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CHART XVIII 
INDICATED FIELDS OF INTEREST OF GOOD AND Poor STUDENTS 


Explanation: This chart is based on the replies of students who ranked in either the 


ee 
hiohes 
hignest 


or the lowest tenth of the distribution of students in each college on the General 


Culture Test. The numbers at the left indicate the number of times each field of interest was 


checked by these students. 
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ional personnel services are usually quite ex- 
pensive. While there is no doubt of the value 
of such services when available, it is perhaps 
hot recognized generally enough that a great 
deal of effective guidance can be given by 
regular instructors and deans, and that with 
some slight assistance, college students can 
make intelligent decisions themselves on the 
asis of knowledge of their test scores. 

The primary requisites for student. self- 
guidance are a battery of tests covering a 





sufficient variety of fields, a system for ex- 
pressing the scores on these tests in compar- 
able units, and some means for helping the 
examinee to picture the pattern of his own 
scores. In planning the 1939 National Sopho- 
more Testing Program, definite efforts were 
made to meet all three of these requirements. 

The first step consisted in the broadening 
of the basic battery of tests. This was done 
in several ways. The General Culture Test 
had been revised the year before to include 
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CHART XIX 
COMPARISON OF SCORES ON THE CONTEMPORARY AFFAIRS TEST OF GOOD AND Poor ee 
Explanation: This chart shows the profiles of average scores on the subtests of the = 
temporary Affairs Test of men and women who ranked in either the highest or the lowest +» * 
per cent of the distribution of students in each college on the General Culture Tes; _ SPECIAL 
averages are plotted in terms of national percentiles as in Chart I. 7" 
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sections on science and mathematics, as well edge of current happenings—one in the fields 
as on social studies, foreign literature, and of political, social, and economic events, and 
fine arts. This year it was further revised by _ the other in the areas of literature, music, an 
making the literature test cover more or less art. In the 1939 edition, six different su’ are 
the whole field of world literature. The tests were included. The first two covered tren le 
greatest change in the test battery however, government and politics, and social and ec 
came in the revision of the Contemporary nomic events. The third dealt with current 
Affairs examination. This test had formerly happenings in the fields of science and mec 
provided only two general measures of knowl- icine. The fourth and fifth covered contempe 
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CHART XX 


Explanation: This is a copy of the Individual Profile Chart, filled out with actual data 
trom the scores of a student in one of the colleges participating in the Sophomore Program. 


opeCLAL PROFILE CHART FOR COOPERATIVE TEST SCORES: 1935 NATIONAL COLLEGE SOPHOMORE TESTING PROGRAM 
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For Students Who Recorded Responses to General Culture and Contemporary Affairs Tests in the Booklets 


CONTEMPORARY AFFAIRS 
aterr ’ 








PERCEN 











CTIONS. It is suggested that each student fill in his own test results 
‘s Chort. For the English and Literary Comprehension Tests, he should 
cles around the dots in the appropriate columns, at the levels of his 
sed Scores in these subjects. The Scaled Score values are shown in the 
me et the left of the Chart. For the General Culture and Contemporary 
ans T t is only necessary to encircle the appropriate raw score in 
Other tests which yield Scaled Scores may be entered in the 
speces provided, and the correct dots encircled by reference to 


J Score values in the column at the right of the Chart. 
" 








percentiles may be entered in the spaces at the extreme right, 
percentiles. if these have been computed, in the spaces at the 
ere left of the Chart, to facilitate further comparisons. 


COOPERATIVE TEST SERVICE 
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VALUE OF THE CHART. This Chart enables the student to appraise his 
own performances in various ways: 


1. He can compare his score on any test with the Scaled Score 50, 
which is the score thet the “average American” (1.0. 100) would be expected 
to make upon graduating from an average high school. 

2. He can compare his score on any test with the scores of sophomores 
in his own and in other institutions by means of the local and national 
percentiles 

. He can study his own relative achievements and functioning interests 
in different fields by direct visual inspection of his profile, since all tests ere 
equated to the Scaled Score units. This is perhaps the most importent use 
of the Chart, and in fact the most important purpose of the National 
Sophomore Testing Program, since it gives the student direct eid in ep- 
praising his own particular strengths and woaknesses. 


NEW YORK, N. Y, 
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rary literature and fine arts; and a sixth sec- 
tion was added dealing with radio, moving 
pictures, and sports under the general head- 
ing of amusements. All of the items making 
up this test were on events of the past year 
only, so that the scores on the various sec- 
tions should indicate the student’s function- 
ing interests in the corresponding fields, 
rather than his specific instruction in college. 
While the total score on a test such as this 
seems to show primarily the general alertness 
of the individual, it was felt that the pattern 
of scores on the six subtests might reflect the 
corresponding interest pattern as this reveals 
itself in information actually learned and re- 
tained. If the Contemporary Affairs Test pro- 
file does reflect the pattern of these function- 
ing interests, the use of this test should add 
considerably to the value of a guidance bat- 
tery based on tests the rest of which are 
measures of school achievement primarily. 


Given an adequate battery of tests, the 
utilization of their scores by the students 
who take them demands some simple and 
comparable system for expressing these 
scores. A number of the Cooperative tests 
have been provided with a special system of 
Scaled Scores. Among these are the English 
and Literary Comprehension examinations. 
The Scaled Scores are directly comparable 
from one form of a test to another and from 
one examination to another. While the Gen- 
eral Culture and Contemporary Affairs Tests 
have not been scaled directly as yet, approxi- 
mate Scaled Score Equivalents were derived 
as early as possible on the basis of about the 
first 5000 sets of scores sent in. These Scaled 
Score Equivalents were obtaired by consider- 
ing the percentiles on the various subtests of 
the General Culture and Contemporary 
Affairs examinations equivalent to the cor- 
responding total Scaled Score percentiles on 
the English Test. 


With comparable scores provided for all 
the tests of the minimum recommended pro- 
gram, it was still necessary to devise some 
method of making them available to the indi- 


vidual students in an easily interpreted fo, 
A special profile chart was prepared { thi 
purpose, and copies of this chart were 4. 
tributed to all the participating colleges ;. 
numbers sufficient so that one could be give 
to every individual student. Chart XX < -.. 
produced directly from one of these py: 
charts, filled out with actual data from the 
scores of a student in one of the participatin: 
colleges. The range of scores of this sty¢ 
is tremendous, varying from a spelling sco, 
below the fifth percentile (and just aboy 
average for the eighth grade) to a sciens 
score above the goth percentile for the tots! 
sophomore group. Since this student's math. 
ematics score is almost as high as his scienre 
score, and since his science and medicine 
score on the Contemporary Affairs Test js 
also his highest in this set, it would appear 
that his professional goal of engineering ha: 
been wisely chosen. This finding is corrobp- 
rated further when we note that his maj 
field of study as an undergraduate is mathe. 
matics, he believes course lectures to haw 
been the major contributor to his cultural 
development, he prefers the Scientific Amvr- 
ican to other magazines, and he lists scien. 
tific interests and activities as appealing : 
him most. 

It is to be hoped that eventually college 
will come to realize that the greatest value o/ 
any testing project—as of any other phase o! 
the educational program—is its value to th 
individual student. The major purpose o! 
higher education in a democratic society is t 
assist as many members of the oncoming 
generation as possess the requisite background 
in the development of their potentialities for 
effective personal living and for useful service 
to society. The purpose of guidance is to hel; 
each student to appraise his abilities, inter- 
ests, and background; to plan his further edu: 
cation intelligently in the light of this ap 
praisal; and to select and prepare himself ior 
a vocation which will be suited to his poten- 
tialities, his limitations, and his interests. 
Guidance and instruction are twin aspects 0! 
any adequate educational program. 
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CORRECTIONS TO CORRELATION COEFFICIENTS 


ON ACCOUNT OF HOMOGE 


TEOBALDO ( 


NEITY IN ONE VARIABLE 


“ASANOVA 


Department of Education of Puerto Rico 


Very often it is necessary to use a sample 
‘hat includes several grades or several age 
evels in order to find the correlation between 
syn other traits, say XY and ¥. This may 
happen because the sample in one grade or 
ave class is too small to determine the cor- 
relation with the desired degree of accuracy. 
If the correlation in the large range is known, 
in estimate of the correlation in the small 
range may be had through the use of well 
known formulas, provided the standard devi- 
tion of one of the variables in the small range 
is available. The partial correlation technique 
is not applied here either because the number 
{classes in the third variable is usually small 
r because the regressions are not linear. 

Sometimes the situation is reversed. The 

rrelation of XY with Y within a grade or a 
single age class is known and it is desired to 
estimate the correlation in a range of several 
grades or age Classes. The usual procedure is 
to make the necessary assumptions of linear- 
ty and homoscedasticity and apply one of 
the formulas available for the purpose, if the 
standard deviation of one of the variables in 
the large range is known. 

The writer has pointed out some of the 
(ificulties encountered in applying the 
formulas for estimating the correlation in one 
range from that obtained in a different range 
when rectilinearity and homoscedasticity are 
not present in the data.’ In some cases the 
ibsence of these properties from the correla- 
tion chart may be due to the fact that the 
listribution of one variable in the large range 
is homogeneous while that of the other vari- 
able is heterogeneous. As used here, the term 
homogeneity” implies that the means and 


‘Casanova, T. “A test of the assumptions of linearity and 
‘omoscedasticity made in estimating the correlation in one 
-— fre m that obtained in a different range.” J. Exp. Educ. 
sil, 245-249, 1939, 





standard deviations are equal in all the grades 
or ages. This quality was observed while 
finding the correlation between an arithmetic 
test and the school marks in the subject. 
When the marks A, B, C, D and F are turned 
into numerical equivalents, the means of the 
several grades are approximately equal to each 
other, and the standard deviations are almost 
exactly equal, while the test distribution is 
heterogeneous throughout the grades. The 
same situation prevails whenever the same 
rating scale is used separately to classify the 
children in each grade. Thus, homogeneity is 
usually observed when each of several classes 
is rated on the basis of a second trait through 
an identical rating scale, and then the rec- 
ords are mingled into a single total group. 
However, this property may also result from 
direct measurement, as may be observed in a 
chart showing the correlation between two 
traits within a given age range, one of which 
traits continues to grow throughout the en- 
tire age range, while growth in the other trait 
has ceased before reaching the lower limit of 
the age range, development in this trait re- 
maining practically stagnant throughout the 
range. In this case the means of the age sub- 
groups will increase with the age in the grow- 
ing trait, while the means and standard devia- 
tions of the stagnant trait will be approx- 
imately equal in all the subgroups. 

As far as the writer knows, none of the 
existing formulas is suitable for transmuting 
correlations from one range to another in the 
presence of the above described conditions. 
Let R,, be the correlation of X with VY ob- 
tained by mingling the records of m subgroups 
into a total single record. Then, we have,? 


2 For this formula in more explicit form, that is, without 
the summational notation, see Dunlap, J. Combinative prop- 
erties of correlation coefficients. J. Exp. Ed., 1937, V, 286— 
288. 
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in which \,, vx, , ay, and ry, y, are the statist- 
ics of the subgroup 7, d,, is the difference be- 
tween the mean of subgroup 7 for the x- 
variable and the mean of the total group for 
the same variable, and d,, is the correspond- 
ing difference in the y-variable. If the y- 
variable is homogeneous so that M,, == M,, = 
saa aah Oy, and dy, = 


n 
2 [yc 1,0) 
I 








Ryy ig ais \ n } rg (2) 
\ N = [Nilo T dz) ] a 
| f= 7 \ 

where N is the number of cases in the total 


group. This formula gives an estimate of the 
correlation that would be obtained by direct 
calculation in the large group when the sta- 
tistics of the component groups are known. 
Although it has no practical application be- 
cause the means of the x-arrays are not rec- 
tilinear and, therefore, the Pearson “r’” is not 
a fair measure of the existing relationship, it 
may serve to detect the error made in esti- 
mating the correlation in the large range 
through formulas in common use. 


If the correlation in the total group is 
known the correlation in one of the sub- 
groups, or their average, may be estimated. 
The quantity in the bracket in the denom- 
inator is equal to S,, the standard deviation 
of the heterogeneous variable in the large 
range. Now, if all the o,’s are all equal to 





each other, or nearly so, and if V, = N, = 
ame N,, 
n 
= fs 
. oh 
tn a | 
n De 
and 
Ryy Sx 
poe ee, Ain cncnsnmeincenes (4) 
Ox 


where r, is the average correlation in the sub- 
groups. Since x is the heterogeneous variable, 
S,>o,, and r,>R,,. This is an exceptional 
result in view of the fact that correlations in 
wider ranges are usually expected to be larger, 
as evidenced by the formulas for their esti- 
mation in terms of correlations in narrow 
ranges. 
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In case that the x— distribution is thy 

a test given in several grades, the differ rene: FE 

between the mean of one grade and that 

the next higher grade is, in general, pe.;, 

constant. If this difference is assumed ty) 

constant, if its value is equal to A,, and jj ty. 

mean of the total group lies at the mic D - 

of the distribution of the means of the o». 

groups, 


a FRO 4 

i=! Vy 
The fraction on the right is the well know 

formula for the sum of the squares of th 

deviations of the first m natural number 

around their own mean. The above equatio: 

is easily understood by supposing that 

mean of the first m natural numbers has been 

subtracted from each one of them and each 

remainder multiplied by A,. Substituting th 


| a) 


M 


in formula (2), and 

4 =m I A 
making again the assumptions involved in the 
derivation of (3), 


n 
above value of yw d: 
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where once more it appears that r,>X,,. | juanti 
the value of the fraction under the radical ines a 
can not be negative. The value of 7, increases each re 


in comparison to that of R,, as the common One 


difference, A,, increases, and as the number the tol 
of subgroups increases, but the proportion \s openeit 
independent of the unit of measurement oi * tions 1 
as this is a common factor of the numerator — 
and denominator of the fraction under the oper 
radical. , - 

The preceding formulas compare the or pei 
relations in the subgroups with the correla the dis 
tion in the total group when one of the vari: ind th 
ables is homogeneous. The most importa! A. 
problem, however, is to estimate the correl2- hs te 
tion that would be obtained for the tote the qu 
group if both variables were heterogeneous subero 
from the correlation that has been observet 
when one of the variables is homogeneous. 

Fig. 1, slab ABCD, shows three regress" nm 
lines for a hypothetical case in which y is the and it 
homogeneous variable and x is the heterose 2b Ay 


neous variable. The lines represent the regre= ment ¢ 
and my, are 


sions of y on x; and m,,, mx, measut 
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Fig. 1. Shifting several regression lines into rectilinearity. 


‘he means of the subgroups 1, 2 and 3. Here 
mn, = My, == My, == M,, the mean of the total 
group. It is further assumed that m,, — 
M;, = My, — Mz, == A,,, that A, is a positive 
quantity, that the slopes of the regression 
ines are nearly equal to each other, and that 
each regression is linear. 

One way of making rectilinear the data of 
the total group, and thus introducing heter- 
ogeneity in variable y, is by shifting distribu- 
tions 1 and 2 upward until their regression 
ines AL and OP interest line HJ respectively 
at points m', and m,., as shown in Fig. 1. 


Let } stand for the slope of the line HJ. Then 
the distance moved by KL is m,, m',, — b Ax, 
und the distance moved by OP is m= 
“)\;. This amounts to adding the quantity 
* 4, to every y— measure in subgroup 2, and 
‘he quantity 2b A, to every y— measure of 
subgroup 3. But 
5 i a eee 
Ox 

and it is seen that the increments b A, and 
; » 4, are independent of the unit of measure- 
vent of the x— variable. As the scales of 
“asurement are purely arbitrary, the amount 





to be added to the y— measures of subgroups 
2 and 3 may be selected at will by choosing 
the proper scale for the y— variable. If all 
the measures are expressed in standard scores, 
b ==r,y, and the increase in y per unit « n- 
crease in x depends on the correlation alone. 

Suppose that x represents a test in a certain 
subject and y the school marks in the same 
subject for a group including various grades. 
If the same rating scale, say A, B, C, D and 
F, is used in all the grades, the y— distribu 
tions will be homogeneous, and the means 7nd 
the regression lines will lie as shown in Fig. 1. 
The numerical equivalents assigned to the 
letter-grades are at our disposal, and so is 
A,, the quantity to be added to every measure 
of group 2. But if the latter is expressed in 
terms of oy, it will be independent of the 
numerical equivalents, for if these are multi- 
plied by a constant &, oy will be also multi- 
plied by &. 

In the case of m subgroups, the effect of 
shifting the regression lines, that is, the effect 
of adding the constants b A,, 2b A,------ 
(n —1)b A, to each of the y— measures of 
the groups 2, 3, ----- n respectively, will be: 

1—The y— means of the subgroups 2, 
‘aleieentll n will be respectively increased by b A,, 
. (nm —1)bA,. 
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2—The o,’s will remain unchanged, as add- Let 
ing a constant to each measure does not affect 
the standard deviation. 





























3—-None of the statistics of the x— vari- ied grades 
able alone will suffer any change no matter E were 
what the distance moved in each case. But - Then. 9, and 
the means and standard deviations of the y— 7 .dded 
arrays will be affected as these are functions hes l tatisti 
of both variables. ; ime oe | vere a 
4—-The correlations within the subgroups ts 5 : rn | 
will not be altered by the addition of a con- = , G 
stant. | nv 
5—Since A, is now equal to 6 A,, using (5), - ” 
the last term in the numerator of (1) becomes 
or, 
n(n> — 1) Fe 
oat b AY r = y (dQ) 
I2 On. . 
I —— 
. . b Oo Ot 
6—In like manner, the second bracket in C=. ) ‘ 
the denominator of (1) is now a, ( 
I + 
- a | 
n ; 2 o 
\ ? m(n°—1I) .o,.;% / = ’ _ 
1 S Wifes +———" # a} 
{i Le ne a \ . , 
If x and y are expressed in terms of stand- 
Formula (3) gives the value of the correla- ard scores in the subgroups, 
tion observed in the total group, R,,. Let ms = 
c R,, be the correlation in the total group Fy hol 
after the above corrections for homogeneity , 1 ee Marks 
have been made. Then the equivalent of R,, ( Ry — 
in (3), after correction, is fi+on, 
= = Test 
n 
= (Px, Ox.) ; 7 Now 6 = ryxy, the correlation in subgroup 
oe vw (n*— 1) 4, r Therefore, Bott 
n 120, Se? ; — 
ee i sy (7) "1 Se, 
(m?—1) A’. Bb br. 
S. ae x By 
RZ 7 
In (g) it may be seen that if all the rs 
the equivalent of (4), after correction, is have the same sign, b and r, are of the same The 
(n?—1) Ab sign, and that o,,, must be positive, since pray 
Ox%, +> re is involved in the expression. Theretore, the 
7 = - —¥'______(g) fraction in the last equation is a positive im } 
ie 9 © . nei 
. (n? — 1) A’, 6? proper fraction, and C and R,, have the same 
; “Vy + 120," sign, that is, the correction for homogeneity ‘™ 


always increases the absolute value of the 
and the correction for homogeneity, C, which observed correlation, whether its sign Is pos 











is equal to ¢ Ryy — Ry, is tive or negative. 
—(n®—1) Ab 7 
ox t, + —————> 
C I , . I20 
Se / (n? — 1) A’ b? — 0x% | -------------------------- . 
yr re," 
IZ vy 
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rhe formulas were applied to the correla- 
von of an arithmetic test and the school 
sarks of 293 Children of 6th, 7th and 8th 
se lhe school marks of A, B, C, D, and 
© were assigned equivalents of 4, 3, 2, 1 and 
. and the records of the last three years were 
‘iaed to give a range of from 1 to 12. The 
statistics of the school marks and the test 


Here the average difference in the m,’s was 
put in the place of A,. This result agrees with 
that obtained through formula (3), but both 
fall short of the actual value which is .4r. 
This is due to the assumption that the m,’s 
are equal, which results in eliminating the last 
term of the numerator of (1), thus leading 
to a low estimate. Substituting the average of 


rrades 




















ere as follows: the o,’s in (8), we have 
8 X 116.21 & .10 
45 \ 12.50 +=-— 12 9 5.11 
Ra = = —— -44, 
: ; SX £56.25 X 01 
15.28 / I + 
12 §.11 
V (9 
aq 8 X 110.04 X .10 "7 
45 X 12.50 +- ———__———_- 
12 II . 
( | 2 ——— 65: X 12.50 07, 
15.28 8 & 110.04 X .O1 
[—+- 
| EZ X §.1% | 
= GRADE and by (ga) 
Statistic? 6th 7th 8th All - si 
sinol { my --- 6.04 6.22 6.47 6.27 , 1.52 
nw te, on SO 228 2 ' 5.63 
Marks | ¢ C = .41 |}—————_]— .08 
lark L Dy — 2.26 4 VI IS 
(m, ---56.06 67.99 77.61 = > 
let 4 wy Au EELG «612g 64983 
Sx --- 15-28 The observed value of R., was substituted in 
roup 1 (3 ---80 - 99 as n5q ~ 293 (ga). If instead, the value of R,, calculated 
Both * ai ; 41 by formula (6) is substituted, C — .07, which 
) ae 08 07 agrees with the values of R,, —= .37 and c Ryy 
By formula (3) .44, as obtained above. 
the 1's R, = hse = 39 Formulas (9) and (ga) are the most con- 
he same " — P . venient and accurate to use, since the error 
aiheeay The average of the o,’s was put in the Toy 
since it made in assuming that m, == my = _-_- 
place of oy. By formula (6), : : Pa. |» 
lore, the 45 m,., Which resulted in the elimination of the 
itive im- Ry = 8 X 116.21 = 37 last term of the numerator of (1), is not 
he same A rear ry 36 25 present here, because this term is eliminated 
ogenelty ‘The statistics were first calculated to four decimal places. When R,, is subtracted from c Ryy. 
» ol tne 


} is posi- 











EXPERIMENTAL DESIGN AND STATISTICAL TREATMENT 
IN EDUCATIONAL RESEARCH 


EUGENE SHEN 
Harvard University 


I 


Experiment distinguishes itself from other 
types of observation by the presence of some 
control of the conditions under which obser- 
vations are made. Curiously, the experimen- 
tal factor which is a variable under direct 
control is conventionally never referred to as 
a controlled variable. On the contrary, con- 
trol in the latter term means holding con- 
stant, so that a controlled variable is precisely 
a factor which is not allowed to vary. 


In educational research, direct regulation is 
feasible for only a few of the many factors 
that require control. Most obviously, the 
heredity and previous experience of the re- 
agents are variable beyond the experimenter’s, 
or any human, power, and such variables can 
at best be brought under indirect control by 
means of selection in the form of matching 
between subjects with reference to some 
measurable character. But equating is not 
always feasible between groups receiving dif- 
ferent experimental treatments, and is never 
desirable within such groups. A still more 
indirect method is then substituted whereby 
the differences are merely noted and their 
effects allowed for in the subsequent evalu- 
ation of results. Such adjustment is termed 
statistical, as distinguished from experimental, 
control. 


When all these various methods of control, 
indirect as well as direct, statistical as well 
as experimental, are resorted to, there are 
always other variables which must be left to 
chance. The very necessity of using some sta- 
tistical tests of significance in the evaluation 
of experimental data speaks for the omni- 
presence of such uncontrolled variables. 
While increased control generally entails im- 


1Some direct control is generally implied in the term ex- 
periment. A study of sex differences, for instance, becomes 
experimental only when the character of sex itself is experi- 
mentally induced, or when the two sexes live under experi- 
mentally arranged conditions, or at least when they are 
experimentally assigned to comparable existing conditions. 
Merely matching the groups with respect to biological and 
social background, however adequate for given purposes, is 
not generally accepted as experimental procedure. This con- 
—— distinction is not essential in the following discussion, 
owever. 


proved precision whereby an experiment i 
rendered more efficient, the presence of ever 
large chance factors does not necessari) 
vitiate the validity of a conclusion, provides 
the proper test of significance is employed 
Indeed, the evaluation of the standard erro 
is logically nothing but a case of statistica! 
control for the random effects of unmeasured 
factors. 

The proper evaluation of observed differ. 
ences between groups receiving differen: 
treatments in the light of variations withir 
groups of subjects uniformly treated is based 
upon an assumption of the unmeasured vari- 
ables being randomized with respect to the 
experimental treatments. If an assumed 
random factor operates in a biased manner 
any computation of standard error will be 
idle and misleading, unless the direction and 
amount of bias can be determined and allowed 
for. If, on the other hand, a controlled vari- 
able is treated as if a random variation were 
permitted between the groups, then the sig- 
nificance of the difference will be underesti- 
mated. Such invalid estimation of significance 
is a necessary consequence of incoordination 
between initial design and subsequent treat- 
ment, or a failure to grasp as a unity the di- 
ferent phases of a coherent logical structure 


II 


The most general type of educational prob 
lem is limited to two experimental treat: 
ments; two methods of instruction in a cer 
tain subject, say. Aside from a minimum 
direct control of some factors, such as t 
length and distribution of instruction perio’ 
which will always be necessary, the degree 
which other factors, such as the maturity an 
brightness of the pupils, are subject to match- 
ing and statistical control, is left to the option 
of the investigator. The crudest experiment 
design will leave all variations between pup! 
uncontrolled, and the corresponding statistics 
treatment is then a computation of a critica 
ratio of the following form, 
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where x, and x, refer to individual scores on 
anal achievement, x, and x, to their means, 
, and «, to the standard deviations, and N, 


and V, to the number of pupils, respectively, 


‘the two groups. The interpretation of this 
-atio is made in terms of the probability of 
che observed difference arising out of chance 
variation from an hypothetical difference of 
vero, usually by referring to a table of the 
normal distribution. A slightly different treat- 
ment, which is not yet in general use in the 
field of education, is the ¢ test, based on the 
following ratio, 


biased view is that the more extensively used 
method is a crude approximation never per- 
missible except when samples are large. 
However, a much more serious question 
with reference to the usefulness of either 
formula is whether the uncontrolled variables 
are really randomized. In a laboratory ex- 
periment a randomized assignment of each 
subject to either group can be executed not 
only with ease but with perfect confidence. 
In classroom experiments, on the other hand, 
groups organized upon principles quite con- 
trary to randomization usually have to be 











t  ———— — seal —— 2 ee ee eee esascaeocoe ees eo eo aon onoonocen@ [ 2 ] 
Le 
| N, +N,—2 N,N, 


The ¢ test differs from the prevalent 
method in three respects: (1) The variance 
within groups is given a single determination 
rather than separately estimated for each 
group. Since the very hypothesis to be tested 
postulates the two groups as being random 
samples of the same population, a separate 
estimation of variance within each group is 
clearly unnecessary. (2) In computing the 
variance within groups, the number of de- 
grees of freedom is used in the denominator 
rather than the number of observations. This 
s necessitated by the fact that deviations 
from an observed mean are on the whole 
always less than deviations from the true 
mean, and it can be shown that this discrep- 
ancy is properly corrected for by using the 
number of degrees of freedom, which is the 
number of observations minus the number of 
constants derived. (3) The significance of ¢ 
5 interpreted by referring to a table of the 
‘ distribution, which approximates the nor- 
mal only as the number of degrees of freedom 
‘s indefinitely increased. 


Since in practical importance the difference 
between the two methods varies inversely as 
the size of the samples, those accustomed to 
the less accurate procedure are likely to re- 
gard the ¢ test as an over-refinement useful 
only when samples are small, whereas an un- 


Me For a full treatment of the ¢ test, see Fisher, R. A., Sta- 
al fa for Research Workers. London: Oliver and 





kept intact, and consequently the fundamen- 
tal assumption involved in the use of either 
[x] or [2] hardly ever holds. The objection 
to the use of supposedly random groups, that 
the standard error would be left at a max- 
imum, is therefore of rather secondary impor- 
tance, the primary consideration being 
whether any test of significance is valid at all. 


Ill 


An improved experimental design, and one 
much more generally employed and trusted, 
is that of groups equated by selection on the 
basis of some measured character or char- 
acters which otherwise would disturb the 
effects of the experimental variable. Variables 
most often used as bases of matching are, 
either jointly or separately, age, intelligence, 
and initial status in the field of learning un- 
der experiment. Two groups are usually said 
to be equated when both have the same mean 
and variance. Sometimes, however, equating 
is more meticulously achieved by pairing, 
each to each, the individuals of the two 
groups. Provided that the matched characters 
are closely related to the final achievement 
upon which the relative efficiency of the two 
instructional methods is to be judged, either 
method of equating will remove bias between 
the groups and reduce the actual error vari- 
ance to a considerable extent. 





i 
: 
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The adoption of a design which does away 
with portions of the real errors in the experi- 
ment, be it noted, must be accompanied by 
their elimination in the statistical treatment 
of results. Unless this is realized, the in- 
creased precision becomes worse than useless, 
since, in being inevitably thrown away by an 
inappropriate test of significance, it impairs 
the validity of the conclusion.’ 

For the method of pairing, a correct test 
of significance in place of [1] or [2] is not 
difficult to devise and has indeed been rather 
extensively used. One needs only to compute 
the difference for each pair, make a distribu- 
tion of such differences, and obtain the mean 
and its standard error in the usual manner. 
The critical ratio is thus 


where x,, X,, %,, ¥,, o, and o, have the usual 
meanings as in [1] and [2], 7,, designates 
the correlation between the two groups on 
final achievement, and V the number of pairs. 
The corresponding ¢ test, applicable to small 
as well as large samples, is 


x,—X, 


=(*,— %,— *, + #,)? 
| N(N —1) 


It is interesting to note that the matched 
variables do not enter in either [3] or [4], 
from which it may appear as if the thorough- 
ness with which the subjects were paired 
would be of no consequence. But it is clear 
that the effectiveness of pairing is propor- 
tional to the size of the correlation coefficient, 
which attains a maximum only when all the 
relevant factors are used in pairing. It is un- 
necessary to mention the obvious fallacy of 
pairing the individuals on the basis of rank 
order without equating the groups. 


1It is sometimes maintained that errors of rejecting sig- 
nificant differences as insignificant are less serious than those 
of accepting as significant differences which are really insig- 
nificant. This seems to have been based upon an ill conceived 
application of the principle of parsimony. Two fallacies are 
involved. In the first place, inasmuch as experiments may be 
and often are designed for the confirmation of the obvious as 
well as the exploration of innovations, the direction of con- 
servatism does not always coincide with either tpye of error. 
Then, what is more important, the uncritical application of 
an over- or under-estimated standard error has consequences 
quite other than those of a voluntary adoption of a higher 
or lower level of significance 


Earlier in the discussion it was mentin,. 
without argument that equating within gry. 
is never desirable. Such equating, while ¢#, 
tive in minimizing the standard error. y, 
at the same time minimize the ge nerality 
the conclusion. In other words. the appar 
increase in the significance of results wo, 
be accompanied by a corresponding decr a 
in the range 
method of pairing is powerful to the a 
it is able to reconcile the os contra, 


within groups and a small standard error oj 
difference between them, by a high correly. 
tion between members of pairs. 

The less meticulous method 
groups as wholes is in fact as effective as 


more laborious method of pairing. But owing 
to the frequent use of wrong tests of signif- 
cance, its advantages have not been so obvi- 
ous. Usually formula [1] has been used by 


Though formula [3] appears inapplicabi 
when groups are matched as wholes, it ha: 
occasionally been used. 
sists in deriving regression coefficients for 
final achievement upon the matched variable 
or variables, employing the regression equa- 
tion to make best estimates of final achieve: 
ment for each subject, ranking the subjects 
of each group in the order of their estimate 
pairing the individuals of the t 
groups according to their rank order, using 
some individuals more than once if the tw 
groups are unequal in number, and — 
computing the correlation of observed | 
scores on the basis of such pairing. Whi 
necessarily a makeshift which is at once in- 
elegant and laborious, this method is esse 
tially sound and is to be preferred to a neal 
misapplication of the random- endl formula 


The procedure con- 


But a_ formula especially _ 
matched groups is available.’ 
ation of final scores which is due to vari: ati mn 
in the matched factors has been eliminate’ 
between groups, by experimental control, 
is only logical also to eliminate such variation 


s See Wilks, S. S.. Journal of - Psy: hol f 
205-208. Also Lindquist, 
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eroups, by statistical control. The re- 
ng critical ratio is 


within 


sulll 


X,—%» 
z [5] 
| ay NE ran) t y(t — Mave) 
ay 472 

where the r’s refer to the correlations between 
the matched variable and final achievement 
--ores, for each of the two groups, and the 
ther symbols have the same meaning as be- 
‘ore. The relation between [3] and [5] is 
easily seen if we remember that any correla- 
‘ion between final scores of the paired indi- 


with the matched variable. In other words, 


since 


fyyo = 1.00 and Px,x9 . y= .00 


t follows that 





Vxixe = Txiyi + Txoy2 

Matching is thus seen to have the same power 
s pairing, in rendering the experiment more 
efficient, with the further advantage of rel- 
ative simplicity in practical procedure. 


In the light of discussions of formula [2], 
, minor modification of formula [5] will re- 


luce it to the form of ¢, 


where a single correlation coefficient is deter- 
mined for both groups combined, and the 
number of degrees of freedom is one less than 
in formula [2], since an additional constant, 
r, is computed? 

_ If two or more characters are used as the 
sases of matching, formulae [5] and [6] can 
ve generalized by using multiple as well as 
‘imple correlation coefficients, the number of 
legrees of freedom in [6] to be reduced by 


ne for every additional variable used in 
matching. 


IV 


_The discussion so far has been limited to 
‘é case of two variants of a single experi- 
mental factor. However, it is often con- 


_While this formula and formulae [7] and [8] may appear 
* they are merely applications of Fisher’s methods. No 
will therefore be given. 
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venient, if not necessary, to study in a single 
set-up two or more experimental factors each 
of which may, further, have more than two 
variants. An investigation of the distribution 
of practice, for instance, will probably have 
as experimental variables both the length of 
practice periods and the length of interval 
between them. Since all lengths of practice 
will be assigned to each length of interval, 
the fundamental principle of experimentation, 
that only a single factor should vary at a time, 
is not violated. On the contrary, such a com- 
prehensive design, besides being more econom- 
ical than separate experiments on the two 
factors, makes possible a study of interaction 
between them, which in this instance is un- 
doubtedly very important. 


The statistical treatment most appropriate 
to such experimental designs is the analysis 
of variance. While a full treatment of the 
subject is beyond the scope of the present 
discussion, the method is applicable to so 
many situations, and the concept is so help- 
ful in selecting correct tests of significance, 
that the fundamental ideas will be briefly 
presented.? 


The total variance of a whole series of 
observations is usually divisible into a num- 
ber of independent portions, at least to one 
portion which is the variance between the 


- [6] 


; I I 
~fa)\ N,N 


means of groups, and another which is that 
within groups. If there are two or more ex- 
perimental factors, or if a single factor has 
more than two variants, the variance between 
groups is further analyzable. Likewise, if 
there is matching or pairing between groups, 
the variance within groups is further analyz- 
able. Any two portions of variance can now 
be compared with each other, but the most 
important comparison is always with the 
residual variance ascribable to error. 


Let us consider the simplest case of two 
experimental treatments. The total variance 
is 


S(x, — x)? + 3(x,—x)? 


1The reader is again referred to Fisher. Besides his Sta- 
tistical Methods for Research Workers, see The Design of 
Experiments. London: Oliver and Boyd, 1937. 
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to be divided by VN, + N, — 1 degrees of 
freedom. This is analyzable into two portions, 
the variance between groups, 


N, (2%, —x)? - N. »(%e — x)? 


i ei I I 
(2) / (y+ NW, 


with a single degree of freedom, and the 
variance within groups, 


S(x,—x,)? + X(x,—%,)? 


to be divided by VN, + N, — 
freedom. 

If the two groups are random samples of 
the same population, the whole variance with- 
in groups will be ascribable to error. If the 
method of pairing is used, then the variance 
between the means of pairs, 


= (x, ey ss x, — x)? / : 


to be divided by N — 1 degrees of freedom, 
should be eliminated from the variance within 
groups, and the residual, 


=(x,— x, —x,+ x,)? /2 


to be divided by N — 1 degrees of freedom, 
is the correct estimate of error. If matched 
groups are used, then a portion of the variance 
within groups which is ascribable to varia- 
tions in the matched variable, 


[X(x, —x,)? + 


2 degrees of 


+(x, — xX» yi te 
should be eliminated, and the residual, 
[=(x, ae x,)* + = (x, abs x,)*] | oo rey | 


to be divided by NV, + N, — 3 degrees of 
freedom, is the error variance. 

The reader will have observed that when 
the variance between groups is divided by the 
appropriate error variance, the resulting 
ratio is exactly equal to the square of ¢ as 
defined in [2], [4], and [6]. The ¢ test is 
thus a special case of the almost universally 
applicable method of analysis of variance. It 
also becomes obvious, since any two portions 
of variance may be compared with each other, 
that wrong tests of significance can be cor- 
rectly interpreted. If [2] is used where [4] 
or [6] should be, the real question being 
tested becomes, not whether the experimental 
treatments are more important than chance 
variations, but whether they are more impor- 
tant than individual differences plus chance 
errors. 
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groups discussed in the ro A paragraph : 
can be easily extended to apply to more com, 
prehensive experimental designs. The chief 
difference lies in the greater number of groyns 
and more complex relations among them, | 
one employs, in the problem on distribytioy 
of practice, four different lengths of practic. 
periods each with four different interya\ 
there will be sixteen different treatments with 
fifteen degrees of freedom, or fifteen jnde. 
pendent comparisons among sixteen groups 
The variance between groups can be divided 
as follows: 
lerrees of 

Source of Variation ‘ reedom 
Between lengths of practice periods 
Between lengths of intervals ____ 3 
Interaction between two factors - 9 


Total variance between groups _. 15 


Each of these three portions may be fur. 
ther analyzed. One possible subdivision 


the three degrees of freedom between lengths 


of practice periods, or between lengths of in 
tervals, 
lengths with the last two, a second comparisor 
of the first and third lengths with the second 


ie 


is a comparison of the first two 


and fourth, and a third comparison of the 


first and fourth lengths with the second a: 
third. A more significant treatment is to 


a regression line or curve to the four means 


and divide the variance into a portion due t 
regression and another to deviation from th: 
regression curve. 


The interaction variance can likewise be 


further analyzed. In the above problem, on 
would not expect the effect of different pra 
tice periods to be independent of the differen’ 
intervals between them; i.e., at least some 0! 
the nine independent comparisons would show 


significant differences. If interaction is insiz- 
nificant, that portion of variance may be use: 
Therefore, when in- 


for an estimate of error. 
teraction can be ruled out on a@ prior: grouncs 


an experiment may sometimes save the labor 
and expense of replication and stil! have 4 


valid estimate of error.* 


V 


Associated with the analysis of variance 's $ 


the analysis of covariance, or the determi 
tion of regression, the purpose of which \s' 
evaluate the effects of concomitant variable: 


1 Crutchfield has proposed to use in an experiment 24) 
no two of which are to be treated alike. Se wn 
Psychology, 1938, V: 339-346. 
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=) that they can be eliminated. Formula [6] 
in fact made use of analysis of covariance, 
ny which the proper error variance for 
matched groups is obtained. But the method 
~ often used in situations where a statistical 
,diustment is required for the variance be- 
-ween, as well as within, groups, i.e., where 
nerfect equating between groups is not feas- 
ble. Those who have experience with match- 


nas 
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For the case of a single concomitant vari- 
able, it is obvious that the required adjust- 
ment is of the type 


(Axy)* / Xy* = b;, 3y* = ray Ex" 


where, to be consistent with the preceding 
symbols, x refers to final scores and y to a 
concomitant variable. In the case of two 
groups, the adjusted total variance is thus 


[=(x, —x)(y, —y) + 3(%, — x) (¥. — y)]? 





(x, —x)? + 3(x, — x)? — 





ng groups will have found, especially when 
‘wo or more characters are used in matching, 
that it often necessitates repeated trial and 





=(y, —y)? + 3(y¥.— y)? 


to be divided by V, + N, — 2 degrees of 
freedom, while the adjusted variance within 
groups is 


s(x, —%)? + (2, — 3)? — 2G OW) + 3 — HO — DT 








error, and considerable loss in the total num- 
ver of cases. They will thus have doubted as 
) whether an experiment may not be ren- 
lered more efficient by retaining more cases 
ind allowing for slight inequalities between 
coups... Moreover, there are concomitant 





N(x, ae) x)? af N,(%2 at x)? =e 


=(9, —¥,)? + =(¥2 — 2)? 


to be divided by N, + N, — 3 degrees of 
freedom. The variance between groups cannot 
be directly adjusted, but the required result 
is obtained by subtracting the adjusted vari- 
ance within groups from the adjusted total 
variance, thus yielding 


[3(x, — x)(y, — y) + 3(*, —%)(y. — y)]*_ 





[3(x, — x.) (¥, — 1) + 3(%, — 2) (92 — 2) ]? 





(¥, — vy)? + 3(¥, — y)? 








variables which may not be desirable to 
equate under certain circumstances. A tech- 
nique for the adjustment of inequality be- 
tween groups is thus very useful. 








=(9, wide ¥,)? + =(y, ane ¥2)? 


This divided by the adjusted variance within 
groups is the variance ratio or f*, which can 
be reduced tc the following form:? 


[ (x, — x,) eee d(y, oe Yo) |? 





(x, — x)? + 3(x, — ay)? — 8 [3(, — 1)? + 302 — 92)" 





N,+N.—3 


f I 





UN 


where 


: 
1 N, 


I + (y, — y,)? = ) _ 
=(y, aia y:)? + =(y, iain ¥2)? 4 








---{7] 


ais > (zx, —x,)(y, a y;) os % (x, — X2) (Vo — 92) 





¥(¥, — 91)? + 3(¥2 — 2)? 





P he practical difficulties of matching will be much re- 
wed if the means are equated without also equating the 
‘ncard deviations, which is far less essential. 





1 As stated before, this is merely an application of Fisher's 
methods, even though Fisher never gives such an explicit 
formula. Cf. Goulden, C. H. Methods of Statistical Analysis. 
New York: John Wiley, 1939, pp. 250-253. 





An extension of formula |7]| for the case 
of two or more concomitant variables would 
make it unmanageable and is therefore in- 
feasible. But a generalization becomes at the 
same time a simplification if stated in terms 
of the correlation between final achievement 
and concomitant measurement. The result is 
formula [8]: 


N,+N,.—2—k | (x, 





where RX is the correlation with mean of the 
total as origin and r with the group means as 
origin. The correlations may be multiple or 
simple, and & is the number of concomitant 
measurements made.' 


For the case of two groups, to which most 
educational experiments seem to have limited 
themselves, formula [8] gives a most gener- 
ally applicable test of significance. If the 
groups are assumed to be random samples 
without concomitant measurement, then R, r 
and & will all be zero, and formula [8] re- 
duces to formula |2]. If the groups are 
equated, then R attains a minimum relative 
to r, and formula [8] reduces to formula [6]. 
Slight inequalities between groups will be 
properly allowed for in the value of R, which 
varies as a function of such discrepancies. 
The size of r is an indication of the degree 
of adequacy with which relevant concomitant 
measurements are made. If a perfect r could 
be obtained—without using so many variables 
as to reduce the number of degrees of free- 
dom to zero, and without permitting such ex- 
treme inequality between the groups as to 
render R equal to y—it would mean a perfect 
control of the subjects, and further statistical 
tests of significance would be unnecessary, 
since any difference would then be signifi- 
cant. 

Formula [8] shows that there are five 
factors which may influence the significance 
of a given difference: r, R, k, and the two 
\’s. The greatest significance is achieved by 
minimizing R and k, and maximizing r and 


1 Formulae [7] and [8] have been discussed somewhat 
more fully in the January, 1940, issue of the Harvard Edu- 
cational Review, with a numerical illustration by W. 
Deemer. 


1Experimenters in laboratories who do not bother about 
statistical tests of significance unwittingly assume this ideal 
state of affairs, which is probably never attainable in the 
field of learning. 
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the two N’s. The practical problem for «, 
investigator is two-fold: First, in order : 
maximize r, he should make as many releyos a ad 
concomitant measurements as feasible oo 
point is usually soon reached after whi o} 
increase in the multiple correlation is so slip}: s eat 
that it will not be worth the accompanyin» . 
increase in &. Second, in order to minin 


24 °3(x, - 2.) ' a 


R, relative to r, the two groups sh 
matched so that the means of the con 
measurements are as nearly equal as poss 
This, however, can only be achieved 
ducing one or both of the two .V’s, and, wi 
discrepancies are only slight, further appr 
imation to perfect equality is usually not 
difficult, but detrimental. Another point t! 
should be noted is that an equal divisior 
the total number of cases between the ty 
groups gives a maximum efficiency 
experiment. 
VI 

There remains the consideration that r 
gression of final achievement upon the 
comitant variables may be different for th 
different experimental treatments. This need 
not be assumed in usual experiments w! 
are limited to relatively homogeneous groups 
of subjects. It is logical to expect, however 
that methods which are suitable to younger 
children need not be so to older ones, tha! 
treatments which are very helpful to the ci 
may even prove a handicap to the bright, a 
that a new system of arithmetic or handwrit 
ing which would be distinctly superior 
adopted from the beginning, will probably 
terfere with habits in some pupils who haw 
begun by the old method. When subjects « 
clude a wide range of maturity, brightness 
initial status of learning, therefore, it 
always be instructive not only to determn 
the regression equations separately for t 
groups, but, further, to study the range 
significance in terms of the concomitant vat 
ables. In other words, the concomitant mea 
urements would then, instead of being ©” 
trolled factors, become experimental variab'e 
in addition to the major factor of the directly 
regulated treatments. If the correspon 








r the -eoression coefficients differ considerably be- 
ler -«een the two groups, it will not be surprising 
‘nd one of the two treatments superior in 
-range and inferior in another, while a test 
vhich is only concerned with averages may 
| no significant difference at all. 
[he statistical analysis for such problems 
necessarily be somewhat more compli- 
ted. But the required analysis of covari- 
ind adjustment of variance will involve 
new principle and should occasion no 
; difficulty. To test the significance of 
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differences between the experimental treat- 
ments for all values of the concomitant vari- 
able or variables, it is only necessary to set 
t appropriate to any chosen level of signifi- 
cance and solve for the limiting values or 
boundary curves in terms of the concomitant 
variables.' 

1Cf. Johnson, P. O., and Neyman, J Tests of Certain 
Linear Hypotheses and their Application to some Educational 
Problems’, Statistical Research Memoirs, 1936, Vol. I, pp 
57-93. Although these authors make use of a different test of 


significance, the concept of a ‘region of significanc 


to the ¢ test equally well 


applic 














PERSISTENCE OF ATTITUDES CONCERNING 
CONSERVATION ISSUES' 


A. C. WILLIAMSON AND H. H. REMMERS 


I. THE PROBLEM 


The problem of this study was threefold: 
firstly, to measure persistence of group atti- 
tudes changed by means of defined social 
stimulus material, secondly, to measure the 
respective variabilities of the groups to dis- 
cover whether they become more or less 
homogeneous after the presentation of such 
material and thirdly, to compare rural with 
urban group-attitudes produced by defined 
social stimulus material and to compare their 
relative homogeneity after the presentation of 
such material. 


II. PROCEDURE 


Four equivalent forms, W, X, Y and Z, of 
the Thomas—Remmers generalized attitude 
scale, Forms A and B to Measure Attitudes 
toward Any Proposed Social Action were de- 
rived by a “split-half” technique, whereby 
Forms A and B were divided equivalently and 
recombined in four different, comparable com- 
binations. Each new form was used to 
measure attitudes toward the following five 
attitude objects. 


1. Allowing the government to tell the 
farmer how to farm. 

2. Allowing each farmer to farm as he 
pleases. 

3. Clean farming. 

4. Taxing all the people to plant new 
forests, and 

5. Draining swamps. 


The scales were administered before any 
stimulus material was read, to 301 rural and 
urban high school pupils in the middlewest. 
The pupils in each high school were divided 
into four randomly selected groups of approx- 
imately equal number and each group was 
measured with a different form (either W, X, 
Y or Z) of the scale. 

_* Tor considerable aid in preparing the manuscript for pub- 
lication we are indebted to Mr. N. T. Schmalzried, Assistant 
in the Division of Educational Reference. 


?McConnell, Robert, “Attitudes Toward Certain Proposed 
Social Actions as Affected by Defined Educational Content,” 
Bulletin of Purdue University, Studies in Higher Education 
XXXI, Further Studies in Attitudes, Series 11, December, 
1936, pp. 100-104. 


Stimulus material,’ intended to chang 
their attitudes in a given direction, i.e., fayor. 
ably toward attitude objects 1, 4 and unfa). 
orably toward objects 2, 3 and 5, was reas 
on September 1 and the same students marked 
a different form of the scales at intervals 9; 
one day (post-test), one month (October 
test), four months (January test) and eigh; 
months (May test, marked by rural group 
only) after the pre-test. 


III. REsutts 


The premeditated shifts® in average atti. 
tude were accomplished in the case of each 
of five issues (Table I). These shifts were 
statistically significant and remained so for 
each group on each attitude object, with 
three exceptions (Table IT), viz., (1) the 
difference between the pre-test and the May 
test of the rural group for attitude object one 
was 1.97 times its standard error; (2) the 
difference between the pre-test and the Jap- 
uary test of the rural group for attitude object 
2 was not statistically significant; (3) the 
difference between the pre-test and the Janv- 
ary test of the urban group for attitude ob- 
ject 4 was 1.32 times its standard error. The 
only statistically significant shift in average 
attitude, made by the urban group, which did 
not persist as changed was for attitude object 
4, taxing all of the people to plant new forests 

Combining the results obtained from the 
rural and urban groups with reference to per- 
sistence of changed attitudes (Table II anc 
Figures 1, 2, 3, 4, and 5) one observes that 4 
statistically significant difference in attitude 
is maintained from the post-test to the Janv- 
ary test for each attitude object considered 
The differences in variability within the tota 
group on the pre-test and subsequent tests 
are generally in the direction of a statistically 
significantly greater variability on post-tests 
(Table III). Important exceptions are ‘0 
attitude object one, allowing the government 
to tell the farmer how to farm, for which the 
difference in standard deviations betwee 


’ The Remmers generalized scale items are arranged io & 
scending order of scale value. A higher value therefore mest 
a more favorable attitude. 
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TABLE II 


DIFFERENCES OF MEANS FOR PRE-TEST AND JANUARY TEST 











Attitude 
Object Rural Urban Total 
Diff. o C. R. Diff. C.R. Diff. o CY} 
diff. diff. diff. in 14 
1.29 . 35 3.69 2.67 . 46 5.74 1. 86 24 7 9: 
l ies : 
et hg . 36 1.97 v 
84 37 2.27 2.15 . 40 5.38 1.38 23 f - 
» ww 
15° 37 2.08 
La . 20 5. 55 1.61 . 30 5. 37 1.31 16 8 19 _ 
1.20° 210 «5.71 ay 
1.32 . 36 3.67 . 49 ot 1.32 . 82 19 4 
4 2 
1. 25* 37 3.38 
; 1,22 25 4.88 3.85 .38 §=:10. 18 2. 40 2210.91 
5 
i. a7" .25 4.68 
*Differences between Pre-test and May Test. 
10 
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a Rural 
14 —_— — — Urban 
13 
- 
i) 
5 : 
=i a 16 
S 
fe] A 
2 Y 
"v Pec co 
Ibe pe tia < 2 
ro L 14 
9 1$§ 
Je 
, = 1 1 ! —t1— 
Pre-test 1 day 1 month 4 months 6 months = 
(Post-test) (October test) (January test) (May test 
Ficure I 


Persistence of Attitude—Allowing the Government to Tell the Farmer How to Farm 
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FIGURE II 
Persistence of Attitudes—Allow the Farmer to Farm as He Pleases 
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FiGcureE III 
Persistence of Attitudes—Clean Farming 
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FIGURE V 
Persistence of Attitudes—Draining Swamps 
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TABLE III 


DIFFERENCES IN STANDARD DEVIATIONS FOR ENTIRE GROUP 


Attitude 


Obiect Post-test 

™ Diff. ¢ om 4 Diff. 
diff. 

A 
vy 18 1.33 49 

A 
2 92 19 4.84 95 

A A 
3 2.52 16 15.75 1.92 

F < < 
e 4 62 14 4.43 15 

E A A 
~  &§ 2.55 18 14.17 2.19 


October test 


January test 


a C.R. Diff. g C.R. 
diff. diff. 
A 
.19 2.58 . 53 . 20 2.65 
A 
.19 5. 00 ia 18 4.28 
A 
.14 13.71 1.47 12 12.25 
< 
.15 1.00 04 .16 . 25 
7 
| 12.88 1.79 16 11.19 


Note: The arrow in each case points to the test which had the larger sigma. 


pre-test. and post-test are insignificant (81 
hances in 100 of a difference greater than 
vero). These chances, however, increase to- 
ward a statistically significant difference to 
») in 100 for both the October and January 


For attitude object 4, taxing all the people 

plant new forests, there is a statistically 
ignificant decrease in variability between the 
ore-test and the post-test. However, this dif- 
erence rapidly disappears on the October and 
january tests, the chances for a difference 
greater than zero going from 75 in 100 to 
; in 100 during the period from October to 
January. 

Rural-urban comparisons disclose statisti- 
lly significant differences in attitude on 
ich attitude object (Table IV). These dif- 
lerences are present only once on both the 
pre-test and the post-test, viz., attitude ob- 
lect 4, taxing all the people to plant new 


forests. 


For attitude object 1 (allowing the govern- 
ment to tell the farmer how to farm) one 
vserves no difference in the attitude of the 
‘wo groups on the pre-test, but after admin- 
stration of the stimulus material, designed to 
shift their attitudes in a direction favoring 
government intervention, the urban group be- 
omes significantly more disposed to allow 
‘he government to tell their rural neighbors 
how to operate farms. There are no statisti- 
cally significant differences in the variability 
of the two groups. 

The urban population proved to be sig- 
nificantly more in favor of allowing the 
‘armer to farm as he pleased, (attitude ob- 
lect 2) than did the farm children themselves 


on the pre-test, but this difference vanished 
after the stimulus material was administered. 
The two groups were practically the same in 
variability. 

The two groups were practically of the 
same attitude on the pre-test with reference 
to clean farming (attitude object 3), but the 
stimulus material caused the rural group to 
become significantly more unfavorable to- 
ward it than the urban group. The urban 
group was less homogeneous after the presen- 
tation of the stimulus material, their stand- 
ard deviations being significantly larger than 
the rural group for post-test, October and 
January tests. This corroborates previous 
findings that with exposure to information 
about an issue, stereotyped attitudes appear 
to break down and become individualized.* 

The urban group was significantly more in 
favor of taxing all the people to plant new 
forests (attitude object 4) on the pre-test 
and on the subsequent tests. 

Measurement of attitude toward draining 
swamps (attitude object 5) revealed no sig- 
nificant differences in average attitude be- 
tween the rural and urban groups on the pre- 
test. On the post-test, October and January 
tests there was a statistically significant dif- 
ference in average attitude with the rural 
group more disposed to drain swamps than 
the urban group, although both were sig- 
nificantly more in favor of retaining swamps 
after presentation of the stimulus material 
(Table I) than they were before. 


_* Remmers, H. H. and Whisler, L. D., “The Effects of a 
Guidance Program on Vocational Attitudes,” Bulletin of 
Purdue University, Studies im Higher Education XXXIV, 
oo Studies in Attitudes, Series 111, September, 1938, pp. 
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rhis difference in average attitude of the 
-,| and urban groups persists through the 
tober and January tests. From some cause, 
yn group becomes more variable in its 

‘ude. as a group, between the post-test 
he October test, for the dispersion of the 
group becomes statistically — signifi- 
larger than that of the rural group. 
ndition is present after the January 


[V. SUMMARY AND CONCLUSIONS 


\ total of 300 rural and urban high school 
sydents were measured, with reference to 
wir attitudes toward five conservation 
sues, one day prior to and immediately fol- 
ving the administration of stimulus mate- 
lesigned to change the average attitude 
{the group in a premeditated direction. The 
jtitudes were remeasured with comparable 
forms of the same scale after one month, four 
months and eight months (rural group only). 
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Comparisons were made of shifts in aver- 
age attitudes; persistence of average attitude; 
changes in variability as affected by the 
stimulus material and differences between 
rural and urban populations in average atti- 
tude and changeability of attitude. 

1. The attitudes of high school pupils to- 
ward certain conservation issues can be sig- 
nificantly changed in a desired direction. 

2. The attitudes of high school pupils to- 
ward certain conservation issues, having been 
changed by defined stimulus material, tend 
to persist as changed after a lapse of as much 
as eight months. 

3. The attitudes of the group were gener- 
ally less homogeneous after presentation of 
the stimulus material than before. 

4. The rural group tends to be less affected 
on the average by the stimulus material than 
the urban group. 


